ANDREOTTI MAYER LOCI AND THE SCHOTTKY PROBLEM 



CIRO CILIBERTO AND GERARD VAN DER GEER 

Abstract. We prove a lower bound for the codimension of the Andreotti- 
Mayer locus N„ i and show that the lower bound is reached only for the hy- 
perelliptic locus in genus 4 and the Jacobian locus in genus 5. In relation with 
the intersection of the Andreotti-Mayer loci with the boundary of the moduli 
space Ag we study subvarieties of principally polarized abelian varieties (B, S) 
parametrizing points b such that S and the translate H(, are tangentially de- 
generate along a variety of a given dimension. 



1. Introduction 

The Schottky problem asks for a characterization of Jacobian varieties among all 
principally polarized abelian varieties. In other words, it asks for a description of 
the Jacobian locus J g in the moduli space A g of all principally polarized abelian va- 
rieties of given dimension g. In the 1960's Andreotti and Mayer (see [2]) pioneered 
an approach based on the fact that the Jacobian variety of a non-hyperelliptic 
(resp. hyperelliptic) curve of genus g > 3 has a singular locus of dimension g — 4 
(resp. g — 3). They introduced the loci N g ^ of principally polarized abelian vari- 
eties (X, Q x ) of dimension g with a singular locus of @x of dimension > k and 
showed that J g (resp. the hyperelliptic locus Tt g ) is an irreducible component of 
Ng^^i (resp. A^ g _ 3 ). However, in general there are more irreducible components 
of Ng t g_4 so that the dimension of the singular locus of Qx does not suffice to char- 
acterize Jacobians or hyperelliptic Jacobians. The locus N g ,o of abelian varieties 
with a singular theta divisor has codimension 1 in A g and in a beautiful paper (see 
[27]) Mumford calculated its class. But in general not much is known about these 
Andreotti-Mayer loci N g ^. In particular, we do not even know their codimension. 
In this paper we give estimates for the codimension of these loci. These estimates 
are in general not sharp, but we think that the following conjecture gives the sharp 
bound. 

Conjecture 1.1. If I < k < g — 3 and if N is an irreducible component of N g ^ 
whose general point corresponds to an abelian variety with endomorphism ring 7, 
then codim^ 9 ( N) > ( fe ^ 2 ) ■ Moreover, equality holds if and only if one of the 
following happens: 

(i) g = k + 3 and N = H g ; 

(ii) g = k + 4 and N = J g . 

We give some evidence for this conjecture by proving the case k = 1. In our 
approach we need to study the behaviour of the Andreotti-Mayer loci at the bound- 
ary of the compactffied moduli space. A principally polarized (g — l)-dimcnsional 
abelian variety (B, S) parametrizes semi-abelian varieties that are extensions of B 
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by the multiplicative group G m . This means that B occurs in the boundary of the 
compactified moduli space A g and we can intersect B with the Andreotti-Mayer 
loci. This motivates the definition of loci Nk(B, S) C B for a principally polarized 
(g — l)-dimcnsional abelian variety (_B,S). They are formed by the points b in 
B such that S and its translate S& are tangentially degenerate along a subvariety 
of dimension k. These intrinsically defined subvarieties of an abelian variety are 
interesting in their own right and deserve further study. The conjecture above then 
leads to a boundary version that gives a new conjectural answer to the Schottky 
problem for simple abelian varieties. 

Conjecture 1.2. Let k € Suppose that (B, S) is a simple principally polarized 

abelian variety of dimension g not contained in N 9i i for all i> k. Then there is an 
irreducible component Z of Nk(B, 5) with codims(Z) = k + 1 if and only if one of 
the following happens: 

(i) either g > 2, k = g — 2 and B is a hyperelliptic Jacobian, 

(ii) or g > 3, k = g — 3 and B is a Jacobian. 

In our approach we will use a special compactification A g of A g (see [29, 28, 5]). 
The points of the boundary dA g = A g — A g correspond to suitable compactifications 
of g-dimensional semi-abelian varieties. We prove Conjecture 1.1 for k = 1 by 
intersecting with the boundary. For higher values of k, the intersection with the 
boundary looks very complicated. 

2. The universal theta divisor 

Let 7r : X g — > A g be the universal principally polarized abelian variety of relative 
dimension g over the moduli space A g of principally polarized abelian varieties of 
dimension g over C. In this paper we will work with orbifolds and we shall identify 
X g (resp. A g ) with the orbifold Sp(2 5 , Z) x I?9\W g x C g (resp. with Sp(2 3 , Z)\M g ), 
where 

H 9 = {(nj) G Mat(ff x g 1 C) ; t = r*, Im(r) > 0} 
is the usual Siegel upper-half space of degree g. The r»j with 1 < i < j < g are 
coordinates on M g and we let z\, z ff be coordinates on C ff . 
The Riemann theta function i?(r, z), given on H g x C s by 

0(t,z) = ^ e" |ra ' m+2m ' z| , 

is a holomorphic function and its zero locus is an effective divisor 9onI 9 x C 9 which 
descends to a divisor on X g . If the abelian variety X is a fibre of tt, then we let 
©x be the restriction of to X. Note that since 9(t, z) satisfies 0(t, —z) = 0(t, z), 
the divisor Ox is symmetric, i.e., t*(6x) = @x, where 1 = — lx ■ X — > X is 
multiplication by —1 on X. The divisor Ox defines the line bundle Ox (Ox), which 
yields the principal polarization on X. The isomorphism class of the pair (X, Ox) 
represents a point ( of A g and we will write ( = (X, Ox)- Similarly, it will be 
convenient to identify a point £ of with the isomorphism class of a representative 
triple (X, Ox,x), where ( = (X, Ox) represents 7r(£) € A g and x <E Ox- 

The tangent space to X g at a point £, with 7r(£) = (, will be identified with the 
tangent space Tx, x ® ^U 9 ,C — ^f,o © Sym 2 (Tx,o)- If C = @x,a;) corresponds 
to the Sp(2g, Z) x Z 2s -orbit of a point (to,Zo) € H s x C 9 , then the tangent space 
Tx g ,i to A" s at £ can be identified with the tangent space to M g x C 9 at (r , z ), 
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which in turn is naturally isomorphic to C 9 ^- 9+1 ^ 2+3 , with coordinates (ciij,bi) for 
1 < i,j < 9 and 1 < ^ < 9 that satisfy = a^,. We thus view the a^'s as 
coordinates on the tangent space to H g at t and the 6/'s as coordinates on the 
tangent space to A or its universal cover. 

An important remark is that by identifying the tangent space to A g at ( = 
(A, Qx) with Sym (Tx,o), we can view the projectivized tangent space P(T_4 = 
P(Sym 2 (T x ,o)) as the linear system of all dual quadrics in P^ 1 = V(T x .o)- In 
particular, the matrix (a^ ) can be interpreted as the matrix defining a dual quadric 
in the space P 9_1 with homogeneous coordinates (b\ : . . . : b g ). Quite naturally, we 
will often use (z\ : . . . : z g ) for the homogeneous coordinates in P 9_1 . 

Recall that the Ricmann theta function $ satisfies the heat equations 

9 Q , Q 

— — ^ = 2-v^T(l + %) ' 



9zi 9zj ' 9nj 

for 1 < i,j < g, where Sij is the Kronecker delta. We shall abbreviate this equation 
as 

$0 J -0 = 27iV=I(l + <f ij -)0T„0, 
where 9j means the partial derivative d/9zj and 9 Tij the partial derivative 9/9rij. 
One easily checks that also all derivatives of 9 verify the heat equations. We refer 
to [38] for an algebraic interpretation of the heat equations in terms of deformation 
theory. 

If £ = (A", Qx, x) G corresponds to the Sp(2g, Z) k Z 2s -orbit of a point (to, z ), 
then the Zariski tangent space to 6 at £ is the subspace of T Xg £ ^ C 9 ^ 9+1 ^ 2+9 
defined, with the above conventions, by the linear equation 



(1) t; ~r^TT~~ir~\ ai: > d i d ^( T o^ z o) + h <9 £ tf(To, z ) = 

in the variables (a^-, bi), 1 < i,j < g, 1 < I < g. As an immediate consequence we 
get the result (see [35], Lemma (1.2)): 

Lemma 2.1. The point £ = (A, Qx , x) is a singular point of Q if and only if x is 
a point of multiplicity at least 3 for Qx ■ 

3. The locus S g 

We begin by defining a suborbifold of 6 supported on the set of points where 
7T|e fails to be of maximal rank. 

Definition 3.1. The closed suborbifold S g of 9 is defined on the universal cover 
M g x C 9 by the g + 1 equations 

(2) #{t,z)=0, dj#(T,z) = 0, 1 //. 

Lemma 2.1 implies that the support of S g is the union of Sing(9) and of the set 
of smooth points of Q where 7T| e fails to be of maximal rank. Set-theoretically one 
has 

S g = {(A, Q x ,x) eQ:xe Sing(9x)} 
and codim;t s (S g ) < g + 1. It turns out that every irreducible component of S g has 
codimension g + 1 in X g (see [8] and an unpublished preprint by Debarre [9] ) . We 
will come back to this later in §7 and §8. 
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With the above identification, the Zariski tangent space to S g at a given point 
(X, @x,x) of X g , corresponding to the Sp(2g, Z)-orbit of a point (to, zq) e M g x C 9 , 
is given by the g + 1 equations 



^2 aijdr^To, z ) = 0, 
, x Ki<j<g 

(3) 

a ij d Tij d k 'd{T , zq) + ^2 b e d£d k d(T , z ) = 0, 1 < k < g 
l<i<j<g i<e<g 

in the variables (a,ij,bt) with 1 <i,j,£ < g. We will use the following notation: 

(1) q is the row vector of length g(g + l)/2, given by (d Tij 9(T , z )), with lexi- 
cographically ordered entries; 

(2) q k is the row vector of length g(g + l)/2, given by (d Tij dkO(To, z )), with 
lexicographically ordered entries; 

(3) M is the g x ^-matrix (did^To, Zo))i<i,j< g - 
Then we can rewrite the equations (3) as 

(4) aV = 0, o- ^ +6- Aft = 0, 07 = 1,- ■-,<?), 

where a is the vector (a,j) of length g(g + l)/2, with lexicographically ordered 
entries, b is a vector in C 9 and Mj the j-th row of the matrix M. 

In this setting, the equation (1) for the tangent space to can be written as: 

(5) a-q* + b-dd(To,z o y =0 

where d denotes, as usual, the gradient. 

Suppose now the point £ = (X, @x,x) in S g , corresponding to (to, zq) € H 9 x C 9 
is not a point of Sing(B'). By Lemma 2.1 the matrix M is not zero and therefore we 
can associate to £ a quadric in the projective space F(Tx, x ) — P(7x,o) — P 9_1 , 
namely the one defined by the equation 

b-M-b* = 0. 

Recall that b = (b±, . . . , b g ) is a coordinate vector on Tx,o and therefore (&i : . . . : b g ) 
are homogeneous coordinates on F(Tx o)- We will say that is indeterminate, if 
£ e Sing(6). 

The vector q naturally lives in Sym 2 (Tx,o) v and therefore, if q is not zero, the 
point [q] e P(Sym 2 (Tx, ) v ) determines a quadric in P 9 " 1 = P(T X , )- The heat 
equations imply that this quadric coincides with Q^. 

Consider the matrix defining the Zariski tangent space to S g at a point £ = 
(X } @x,x). We denote by r := the corank of the quadric Q^, with the convention 
that = g if £ € Sing(O), i.e., if is indeterminate. If we choose coordinates on 
C 9 such that the first r basis vectors generate the kernel of q then the shape of the 
matrix A of the system (3) is 



(6) A 



q r 0g 
V* B J 
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where q and qt are as above and B is a (g — r) x (/-matrix with the first r columns 
equal to zero and the remaining (g — r) x (g — r) matrix symmetric of maximal 
rank. 

Next, we characterize the smooth points £ = (X,Qx,x) of S g . Before stating 
the result, we need one more piece of notation. Given a non-zero vector b — 
(£>i, . . . ,b g ) G Tx,o, we set db = J2e=i bide- Define the matrix <9&M as the g x g- 
matrix (didjdb'&(To, 2o))i<i,j<cr Then define the quadric dbQ^ — Q^,b of P(Tx,o) 
by the equation 

z ■ d h M ■ z* = 0. 

If z = ei is the i-th vector of the standard basis, one writes diQ^ = Q^j instead of 
Q^.ei for i = 1, ... ,g. We will use similar notation for higher order derivatives or 
even for differential operators applied to a quadric. 

Definition 3.2. We let be the linear system of quadrics in F(Tx,o) spanned by 
and by all quadrics Q^b with b € ker(Q^). 

Since has corank r, the system is spanned by r+ 1 elements and therefore 
dim(Q^) < r. This system may happen to be empty, but then is indeterminate, 
i.e., £ lies in Sing(O). Sometimes we will use the lower suffix x instead of £ to denote 
quadrics and linear systems, e.g. we will sometimes write Q x instead of Q^, etc. 
By the heat equations, the linear system is the image of the vector subspace of 
Sym 2 (Tx, ) v spanned by the vectors q, qi, . . . , q r . 

Proposition 3.3. The subscheme S g is smooth of codimension g + 1 in X g at the 

point £ = (X, Qx j x) of S g if and only if the following conditions are verified: 

(i) £ <^ Sing(O), i.e., is not indeterminate and of corank r < g; 

(ii) the linear system has maximal dimension r; in particular, if b\, . . . ,b r 
span the kernel of Q^, then the r + 1 quadrics Q^, Q^.bn ■ ■ ■ ,Q^,b r are 
linearly independent. 

Proof. The subscheme S g is smooth of codimension g + 1 in X g at £ if and only if 
the matrix A appearing in (6) has maximal rank g + 1. Since the submatrix B of 
A has rank g — r, the assertion follows. □ 

Corollary 3.4. If is a smooth quadric, then S g is smooth at ^ — (X, @x,x). 

4. Quadrics and Cornormal Spaces 

Next we study the differential of the restriction to S g of the map ir : X g — > A g 
at a point £ = (X, Qx , x) E S g . We are interested in the kernel and the image of 
dir\s g ,£- We can view these spaces in terms of the geometry of P 9 ^ 1 = P(Tx,o) as 
follows: 

n ? = F(kev(d7T lSg<( )) C P(Tx, ) 
is a linear subspace of P(Tx,o) and 

S 4 = ¥(lm(d7T lSg ,^) C P(Sym 2 (Txo) V ) 

is a linear system of quadrics in P(Tx.o)- 

The following proposition is the key to our approach; we use it to view the 
quadrics as elements of the conormal space to our loci in the moduli space. 

Proposition 4.1. Let £ = (X, Qx,x) be a point of S g . Then: 
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(i) II{ is the vertex of the quadric . In particular, if £ is a singular point of 
0, then IIj is the whole space F(Tx,o); 

(ii) contains the linear system Q^. 

Proof. The assertions follow from the shape of the matrix A in (6). □ 

This proposition tells us that, given a point £ = (X, Qx,x) G S g , the map dn\s g ,£ 
is not injective if and only if the quadric is singular. 

The orbifold S g is stratified by the corank of the matrix (didj6). 

Definition 4.2. For < k < g we define S g ^ as the closed suborbifold of S g 
defined by the equations on M g x C 9 

#(t,z) = 0, dj&{T,z) = 0, (j = l,...,g), 

rk((didji)(T, z))i<i,j< g ) <g~k. 

Geometrically this means that £ G S g ,k if and only if dim(IIj) > k — 1 or 
equivalently has corank at least k. We have the inclusions 

Sg = Sgfi D SgS D . . . D Sg,g = S g D Smg(6) 

and S g s is the locus where the map dTT\ Sg £ is not injective. The loci S gt k have been 
considered also in [16]. 

We have the following dimension estimate for the S 9y k- 

Proposition 4.3. Let 1 < k < g — 1 and let Z be an irreducible component of S gi k 
not contained in S 9i k+i- Then we have 

coding (Z) < + 

Proof. Locally in a neighborhood U in S g of a point z of Z\S gy k+i we have a 
morphism / : U —* Q, where Q is the linear system of all quadrics in P 9-1 . The 
map / sends £ = (X,Qx,x) G U to Q^. The scheme S g ^ is the pull-back of the 
subschemc Qk of Q formed by all quadrics of corank k. Since codimg(Qfe) = C^ 1 ), 
the assertion follows. □ 

Using the equations (7) it is possible to make a local analysis of the schemes 
S g ^k, e.g. it is possible to write down equations for their Zariski tangent spaces (see 
§6 for the case k = g). This is however not particularly illuminating, and we will 
not dwell on this here. 

It is useful to give an interpretation of the points £ = (X, Qx,x) G S g ^ in terms 
of singularities of the theta divisor &x- Suppose that £ is such that Sing(6jsf) 
contains a subscheme isomorphic to Spec(C[e]/ (e 2 )) supported at x. This subscheme 
of X is given by a homomorphism 

O x , x - C[e]/(e 2 ), / i ► f(x) + A^f(x) ■ 6, 

where A^ is a non-zero differential operator of order < 1, hence A^ 1 ) = db, for 
some non-zero vector b G C 9 . Then the condition Spec(C[e]/(e 2 )) C Sing(0x) is 
equivalent to saying that d and db'd satisfy the equations 

(8) f(T ,z )=0, d j f(T ,z ) = 0, l<j<9, 

and this, in turn, is equivalent to the fact that the quadric is singular at the 
point [b]. 
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More generally, we have the following proposition, which explains the nature of 
the points in S 9t k for k < g. 

Proposition 4.4. Suppose that x e Sing(Ox) does not lie on Sing(6). Then 
Sing(Gx) contains a scheme isomorphic to Spec(C[ei, . . . , £k]/{ti£j '■ 1 < « , J < fc < 
g)) supported at x if and only if the quadric has corank r > k. Moreover, the 
Zariski tangent space to Sing(6jsc) at x is the kernel space of Q^. 

Proof. With a suitable choice of coordinates in X, the condition that the scheme 
Spec(C[ei, . . . , efc]/(ejej : 1 < i, j < k < g)) is contained in Sing(0x) is equivalent 
to the fact that the functions z? and did for i = 1, . . . , k satisfy (8). But this the 
same as saying that didj$(T , z ) is zero for i = 1, . . . , k, j = 1, . . . , g, and the 
vectors e,, i = l,...,k, belong to the kernel of Q^. This settles the first assertion. 

The scheme Sing(0x) is defined by the equations (2), where r is now fixed 
and z is the variable. By differentiating, and using the same notation as above, 
we see that the equations for the Zariski tangent space to Sing(Ox) at x are 
Yli=i bididjfifo, zo), j = l,...,g i.e., b ■ M = 0, which proves the second as- 
sertion. □ 



5. CURVI-LINEAR SUBSCHEMES IN THE SINGULAR LOCUS OF THETA 

A 0-dimensional curvi-linear subscheme Spec(C[t]/(i 7V+1 )) Clof length N + 1 
supported at x is given by a homomorphism 

N 

(9) 6 : O x , x - C[i]/(^ +1 ), / ~ Yl A °' ) /( a; ) ' 

3=0 

with A( J ) a differential operator of order < j, j = 1, . . . , N, with non-zero, 
and A(°) (/) = f(x). The condition that the map 5 is a homomorphism is equivalent 
to saying that 

k 

(10) A«(/ s )^AW/.A(M ffj k = 0,...,N 

r=0 

for any pair (/, g) of elements of Ox ,x ■ Two such homomorphisms S and 5' define 
the same subscheme if and only if they differ by composition with a automorphism 
of C[t]/(t N+1 ). 

Lemma 5.1. The map S defined in (9) is a homomorphism if and only if there exist 
translation invariant vector fields D\, . . . , Dpi on X such that for every k — 1, . . . , N 
one has 

(11) A «= ]T — Lnju...^. 

h 1 +2h 2 +...+kh k =k>0 1- k ' 

Moreover, two N-tuples of vector fields (Di, . . . ,Dn) and (D[, . . . ,D' N ) determine 
the same 0-dimensional curvi-linear subscheme of X of length N + 1 supported at 
a given point x E X if and only if there are constants a,.. .,Cn, with c\ ^ 0, such 
that 

i>: y,<i <- : /),. , i v. 
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Proof. If the differential operators A^- k \ k = 1, . . . , N, are as in (11), one computes 
that (10) holds, hence S is a homomorphism. 

As for the converse, the assertion trivially holds for k = 1. So we proceed by 
induction on k. Write A( fe ) = J2i=i > wnere is the homogeneous part 

of degree i, and write Dk instead of D[ k \ Using (10) one verifies that for every 
k = 1 , . . . , N and every positive i < k one has 

3 = 1 

Formula (11) follows by induction and easy combinatorics. 

To prove the final assertion, use the fact that an automorphism of C[t]/ (t N+1 ) 
is determined by the image c\t + C2t 2 + . . . + CNt N of t, where c\ ^ 0. □ 

In formula (11) one has h k < 1. If A^ = D 1 then A^ 2) = \D\ + D 2 , A^ = 
(1/3!)D? + (l/2)Di£> 2 + D 3 etc. 

h ■ hi 

Each non-zero summand in (11) is of the form (l/hi 1 ! • • • hi e l)D i ^ 1 ■ ■ ■ D i 1 , where 
1 < i\ < . . . < ii < k, i\hi 1 + . . . + i(.hi t = k and h^,. . . , hi e are positive integers. 
Thus formula (11) can be written as 

{h H ,...,h ie } 

where the subscript {h^ , . . . , h ie } means that the sum is taken over all ^-tuples of 
positive integers (h^, . . . , h ie ) with 1 < i\ < ■ ■ ■ < in < k and i\h ix + ■ ■ - + i£h ie = k. 

Remark 5.2. Let x € X correspond to the pair (r , z ). The differential operators 
A( fe ), k = l,...,N, defined as in (11) or (12) have the following property: if / is a 
regular function such that AW f satisfies (8) for alH = 0, . . . , k — 1, then one has 
A( fe )/(r ,zo) = 0. 

We want now to express the conditions in order that a 0-dimensional curvi-linear 
subschemc of X of length N + 1 supported at a given point x £ X corresponding to 
the pair (t , z ) and determined by a given JV-tuple of vector fields (Di, . . . , Dn) 
lies in Sing(6x)- To do so, we keep the notation we introduced above. 

Let us write Di — J2e=i Vadg , so that Di corresponds to the vector r\i — 
(rjn, . . . , rji g ). As before we denote by M the matrix (didj9(T , z ))- 

Proposition 5.3. The 0-dimensional curvi-linear subscheme R of X of length 
N+l, supported at the point x £ X corresponding to the pair (r , z ) and determined 
by the N '-tuple of vector fields (D\, . . . , Djy) lies in Sing(9x) if and only if x £ 
Sing(6j>f ) and moreover for each k = 1, . . . , N one has 

(13) E r~T~~r , ^■^■■^^ = 0. 

where the sum is taken over all i-tuples of positive integers (h^, . . . , h it ) with 1 < 
i\ < ■ ■ ■ < it < k and iih il + • • • + ieh ie = k. 

Proof. The scheme R is contained in Sing(6jss: ) if and only if one has 

A( fe )0(r o ,zo) -0, d ] A^0(T O ,z o ) = O k = 0, . . . , A, j = l,...,g. 
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By Remark 5.2, this is equivalent to 

6(t ,z ) = 0, d j A^e(T ,z ) = k = 0,...,N, j = l,...,g. 
The assertion follows by the expression (12) of the operators A^ fe '. □ 

For instance, consider the scheme R\, supported at x e Sing(9 x ), corresponding 
to the vector field D x - Then i?i is contained in Sing(6x) if and only if 

(14) 77i • M = 0. 

This agrees with Proposition 4.4. If R2 is the scheme supported at x and corre- 
sponding to the pair of vector fields (D±, D2), then R2 is contained in Sing(Ox) if 
and only if, besides (14) one has also 

(15) (l/2) m -d m M + r]2-M = 0. 

Next, consider the scheme R3 supported at x and corresponding to the triple of 
vector fields (D\, -D2, D3). Then R3 is contained in Sing(Ox) if and only if, besides 
(14) and (15) one has also 

(16) (l/3!)r?i • df n M + (l/2) m ■ d m M + 773 ■ M = 

and so on. Observe that (13) can be written in more than one way. For example 
T]2 ■ d m M = r\\ ■ d m M so that (16) could also be written as 

(1/31)77! • d^M + (1/2)77! -d m M + m -M = 0. 

So far we have been working in a fixed abclian variety X. One can remove 
this restriction by working on S g and by letting the vector fields D\, . . . , Dm vary 
with X, which means that we let the vectors rji depend on the variables Tij. Then 
the equations (13) define a subscheme S g (D) of Sing(6) which, as a set, is the 
locus of all points £ = (X,Qx,x) e S g such that Sing(Ox) contains a curvi-linear 
scheme of length N + 1 supported at x, corresponding to the iV-tuplc of vector 
fields D = (Di, . . . , Z?at), computed on X. 

One can compute the Zariski tangent space to S g (D) at a point £ = (X, &x, x ) 
in the same way, and with the same notation, as in §3. This gives in general 
a complicated set of equations. However wc indicate one case in which one can 
draw substantial information from such a computation. Consider indeed the case 
in which D\ = . . . = Dm 7^ 0, and call b the corresponding tangent vector to X 
at the origin, depending on the the variables r^. In this case we use the notation 
DbM — (Di, . . . , Dm) and we denote by R x ,b,N the corresponding curvi-linear 
scheme supported at x. For a given such D = (D\, . . . , Dm), consider the linear 
system of quadrics 

^(D)=F(Im(dTr lSgiDhi ) ± ) 

in P(Tx,o)- One has again an interpretation of these quadrics in terms of the normal 
space: 

Proposition 5.4. In the above setting, the space (D&, m) contains the quadrics 
dbQz, ■ ■ ■ , d b Q^. 

Proof. The equations (13) take now the form 

O(T,z) = O,di0(T,z) = O, 7 = 1,. ..,5 

b ■ M = b ■ d b M = ■ ■ ■ = b ■ d^^M = 0. 

By differentiating the assertion immediately follows. □ 
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6. Higher multiplicity points of the theta divisor 

We now study the case of higher order singularities on the theta divisor. For 
a multi-index I — (ii,...,i g ) with ii,...,i g non-negative integers we set z 1 = 
z'i ■ ■ ■ z l g and denote by di the operator d] 1 ■ ■ ■ d g g . Moreover, we let |/| = 5Zf =1 it, 
which is the length of I and equals the order of the operator di . 

(r) 

Definition 6.1. For a positive integer r we let S g ' be the subscheme of X g which 
is defined on M g x C 9 by the equations 

(17) d I ti(T,z) = 0, \I\ =0,...,r-l. 

One has the chain of subschemes 

. . . c s g r) c . . . c s g 3) c s g 2) =s g c s g v = © 

and as a set S g r ^ = {(X, Oi,i) £0:i has multiplicity > r for Ox}- One denotes 
by Sing^(Ox) the subscheme of Sing^x ) formed by all points of multiplicity at 
least r. One knows that S g r ^ — as soon as r > g (see [36]). We can compute the 
Zariski tangent space to S g r ^ at a point £ = [X, Ox, x ) m the same vein, and with 
the same notation, as in §3. Taking into account that 9 and all its derivatives verify 
the heat equations, we find the equations by replacing in (3) the term 9(t , z ) by 
diO(T ,z ). 

As in §3, we wish to give some geometrical interpretation. For instance, we have 
the following lemma which partially extends Lemma 2.1 or 3.3. 

Lemma 6.2. For every positive integer r the scheme S g r+2 ^ is contained in the 
singular locus of S g r ^ . 

Next we are interested in the differential of the restriction of the map ir : X g — > A g 
to S g ^ at a point £ = (X, @x,%) which does not belong to S g r+1 \ This means that 
Ox has a point of multiplicity exactly r at x. If we assume, as we may, that x is 
the origin of X , i.e. z = 0, then the Taylor expansion of 6 has the form 

oo 

where i?, is a homogeneous polynomial of degree i in the variables z\ , . . . , z g and 
r = X] ■ , 1 ■ ] di6(T ,z )z I 

I=(h,...,i g ),\I\=r 

is not identically zero. The equation 9 r — defines a hypersurface TC^ of degree r 
in Pf -1 = ¥(Tx,o), which is the tangent cone to Ox at x. 

We will denote by Vert(TCj) the vertex of TC^, i.e., the subspace of P 9_1 which 
is the locus of points of multiplicity r of TC^. Note that it may be empty. In case 
r = 2, the tangent cone TC^ is the quadric introduced in §3 and Vert (TC^) is 
its vertex 11^. 

More generally, for every s >r, one can define the subscheme TC^ = TC^ of 
ps-i = ¥{T x ,o) defined by the equations 

9 r = . . . = 6 S = 0, 

which is called the asymptotic cone of order s to Ox at x. 
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Fix a multi-index J = (ji, . . . ,j g ) of length r — 2. For any pair (h,k) with 
1 < h, k < g, let J<h,k) be the multi-index of length r obtained from J by first 
increasing by 1 the index jh and then by 1 the index jk (that is, by 2 if they 
coincide). Consider then the quadric Qj in ps -1 = P(Tx,o) defined by the equation 

d J(hk) 6(To,z )z h z h = 

\<h,k<g 

with the usual convention that the quadric is indeterminate if the left hand side is 
identically zero. This is a polar quadric of TC^, namely it is obtained from TC^ 
by iterated operations of polarization. Moreover all polar quadrics are in the span 
{Q£, \J\ = r — 2). We will denote by Q£' the span of all quadrics , \ J\ = r - 2 
and dbQ^, \ J\ = r — 2, with equation 

d b dj (hk) 6(T ,z )z i z j =Q 7 

l<ij<9 

for every non-zero vector fceC 9 such that [b] £ Vert (TC^). 

We are now interested in the kernel and the image of dir, c ( r ) Equivalently 

we may consider the linear system of quadrics = P(Im(d7T| g (r) p"*")) an d the 
subspace nl r ' = P(ker(d7r, ( r ) t )) of ¥(T X o)- The following proposition partly 
extends Proposition 4.1 and 4.4 and its proof is similar. 

Proposition 6.3. Let £ = (X,Qx,x) be a point of Sg"\ Then: 

(i) II^ r ' = Vert(TC^). In particular, if £ £ S g r+1 \ then IL^' is the whole space 

pCTy,o); 

(ii) Xjf' contains the linear system . 

Remark 6.4. As a consequence, just like in Proposition 4.4, one sees that for £ = 
(A, QxjX) the Zariski tangent space to Sing^ (Ox ) at x is contained in Vert (TCj). 

As an application, we have: 

Proposition 6.5. Let £ = (X, &x,x) be a point of a component Z of S g such 
that TC^ is not a cone. Then dim(Q^ ') = g — 1 and therefore the codimension of 
the image of Z in A g is al least g. 

Proof. Since TC{ is a not a cone its polar quadrics are linearly independent. □ 

The following example shows that the above bound is sharp for g = 5. 

Example 6.6. Consider the locus C of intermediate Jacobians of cubic threcfolds 
in A5. These have an isolated triple point on their theta divisor whose tangent 
cone gives back the cubic threefold. The locus C is dominated by an irreducible 
component of S§ for which the estimate given in (i) of Proposition 6.5 is sharp. 
Cf. [6] where Casalaina-Martin proves that the locus of intermediate Jacobians of 
cubic threcfolds is an irreducible component of the locus of principally polarized 
abelian varieties of dimension 5 with a point of multiplicity > 3. 
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7. The Andreotti-Mayer loci 

Andreotti and Mayer consider in A g the algebraic sets of principally polarized 
abelian varieties X with a locus of singular points on Qx of dimension at least 
k. More generally, we are interested in the locus of principally polarized abelian 
varieties possessing a fc-dimensional locus of singular points of multiplicity r on the 
theta divisor. To define these loci scheme-theoretically we consider the morphism 
7r : X g — > A g and the quasi-coherent sheaf on A g 

i—k 

Definition 7.1. For integers k and r with < k < g — 2 and 2 < r < g we define 
N g ,k,r as the support of '. We also set M g _ k _ r = -K*(N gtktr ), a subscheme of both 
S 3i k and S g r \ We write A/ 9jfc and M g _ k for N g ^,2 and M g _ k _ 2 - 

The schemes A^ are the so-called Andreotti-Mayer loci in „4 ff , which have been 
introduced in a somewhat different way in [2]. 

Note that N gt h, r is locally defined by an annihilator ideal and so carries the 
structure of subscheme. Corollary 8.12 below and results by Debarre [11] (see §19) 
imply that the scheme structure at a general point of N g ,o defined above coincides 
with the one considered by Mumford in [27]. 

We now want to see that as a set N g ^, r is the locus of points corresponding to 
(X, @ x ) such that Sing(0jf ) has an irreducible component of dimension k of points 
of multiplicity r for Qx ■ 

Lemma 7.2. Let X be an abelian variety of dimension g and W C X an irre- 
ducible reduced subvariety of dimension n and let ujw be its dualizing sheaf Then 
H°(W,lj w )^(0). 

Proof. Let / : W' — > W be the normalization of W. We first claim the inequality 
h°(W',oj w >) < h"(W,uj w ). To see this, note that by [24], p. 48 ff (sec also [17], 
Exerc. 6.10, p. 239, 7.2, p. 249), there exists a map f*uiw' — * Hom(f*Ow', ^w), 
hence a map 

H°(W',ujw>) -> H°(W,Hom(.UO w >,uJw))- 
Now Ow — * f*Ow* is an injection and therefore H°(W,Ttom(f*Ow' ,^w) maps to 
H (W, Hom(Ow, ^w)) and we thus get a map H°(W, ujw') — * H°(W 7 ujw) which 
is injective as one sees by looking at the smooth part of W. 

Let W be a desingularization of W . According to [22] we have h°(W, fl^) < 

h°(W',oj w ,). Since W maps to X we have ^(W,^) > n. If h°(W,n^) were 
then A" J ff°(T4 / ,r2^) -» H°(W,n^) would be the zero map contradicting the fact 
that W has dimension n. □ 

Corollary 7.3. We have {X,B X ) € N g ^ r if and only if dim(Sing (r) (Q x )) > k. 

Proof. By the previous lemma and Serre duality for a reduced irreducible subva- 
riety W of dimension m in X it follows that H m (W,Ow) (0) and we know 
H k (W, O w ) = (0) for k > m. This implies the corollary. □ 

There are the inclusions 

Ng.k.V ^= Xg .fc.r— 1, Ng kr C Ngk — l^. 
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If p = (m, . . . , n r ) with 1 < m < . . . < n r < g and m + . . . + n r = g is a 
partition of g we write A StP for the substack of A g corresponding to principally 
polarized abelian varieties that are a product of r principally polarized abelian 
varieties of dimensions m, . . . , n r . We write r(p) = r for the length of the partition 
and write A g ^ r ] for the substack D r ( p ) =r A g , p of A g corresponding to pairs (X, Qx) 
isomorphic as a polarized abelian variety to the product of r principally polarized 
abelian varieties. One has the stratification 

A g, [g] c A, [9-1] c • •• c 

We will denote by: 

i) Ilg = UpA g ^p the locus of decomposable principally polarized abelian varieties; 

ii) ^4g NS ' ) the locus of classes of non-simple abelian varieties, i.e., of principally 
polarized abelian varieties of dimension g which are isogenous to a product of 
abelian varieties of dimension smaller that g; 

iii) A^ nd ^' 1 the locus of classes of singular abelian varieties, i.e., of principally 
polarized abelian varieties whose endomorphism ring is larger than Z. 

Remark 7.4. Note the inclusions U g C A ( g NS) C A^ Dd ^ z . The locus U g is reducible 
with irreducible components A 9yP with p running through the partitions g = (i,g—i) 
of g for 1 < i < g/2 and we have codirn^ (*4 gi (i,g-i)) = i(g — i)- In contrast to 

this A® nd ^ z and ^4g NS ' ) are the union of infinitely countably many irreducible closed 
subsets of A g of codimension at least g— 1, the minimum codimension being achieved 
for families of abelian varieties that are isogenous to products of an elliptic curve 
with an abelian variety of dimension g — 1 (compare with [7]). 

We recall a result from [20] and the main result from [12]. 

Theorem 7.5. For every integer r with 2 < r < g one has: 

(i) N g , k , r =<Difk>g-r; 

(ii) Ng t g- r _ r — A g ! r ], i.e., (X , Q x) € N g ,g- ri r is ail r~folcl product. 
Hence, for every integer r such that 2 < r < g, one has the stratification 

N g fi,r ^ NgS.r D ■ ■ ■ D Ng y k,r D ■ ■ ■ D Ng t g- r ^ r = Ag y [ r ], 

whereas for every integer k such that < k < g — 2, one has the stratification 

Ng,k,2 3 Ng yk ,3 3 ■•■ 3 Ng ykyr D . . . D Ng yk ,g-k = Ag^g-k]- 

8. Lower bounds for the codimension of Andreotti-Mayer loci 

The results in the previous sections give information about the Zariski tangent 
spaces to these loci and this will allow us to prove bounds on the dimension of the 
Andreotti-Mayer loci, which is our main objective in this paper. 

We start with the results on tangent spaces. We need some notation. 

Definition 8.1. Let ( = (X,Ox) represent a point in N g ^ y r- By £ gy k, r (() we 
denote the linear system of quadrics P(T^ k ^), where Tj^ gkrt ^ is the Zariski 
tangent space and where we view IP(T4 s { ) as a space of quadrics as in Section 2. 
As usual, we may drop the index r if r — 2 and write Cg t k{Q f° r £g,k,2(()- 
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Notice that 

dim c (7V Sifeir ) < + ^ - dim(£ g , fe «)) + 1. 

Definition 8.2. For C = {X,Q X ) e N gtk , r we denote by Sing (fc ' r) (0 X ) the locally 
closed subset 

Sing^(ex) = {x e Sing(6 x ) : dim x (Single*)) > k}. 
Moreover, we define Q^' r ^ to be the linear system of quadrics in P 9_1 = F(Tx,o) 
spanned by the union of all linear systems with £ = (X, Ox,%) and x G 

Sing( fe ^(e x ). 

Propositions 4.1 and 6.3 imply the following basic tool for giving upper bounds 
on the dimension of the Andreotti-Mayer loci. 

Proposition 8.3. Let N be an irreducible component of N g ^,r with its reduced 
structure. If C — (X,Qx) is a general point of N then the projectivized conormal 
space to N at viewed as a subspace ofP(Sym 2 (T x )), contains the linear system 

Proof. Let M be an irreducible component of ir*N in S^. If £ is smooth point of 
M then the image of the Zariski tangent space to M at ( under dir is orthogonal 
to for all x e Sing^ r (9 x ). Since we work in characteristic the map dir 
is surjective on the tangent spaces for general points m e M and n(m) G N. 
Therefore the result follows from Propositions 4.1 and 6.3. □ 

We need a couple of preliminary results. First we state a well known fact, which 
can be proved easily by a standard dimension count. 

Lemma 8.4. Every hypersurface of degree d < 2n — 3 in P" with n>2 contains a 
line. 

Next we prove the following: 

Lemma 8.5. Let V in V s be a hypersurface of degree d > 3. If all polar quadrics 
of V coincide, then V is a hyperplane H counted with multiplicity d, and the polar 
quadrics coincide with 2H. 

Proof. If d = 3 the assertion follows from general properties of duality (sec [40], p. 
215) or from an easy calculation. 

If d > 3, then the result, applied to the cubic polars of V, tells us that all these 
cubic polars are equal to 3H, where H is a fixed hyperplane. This immediately 
implies the assertion. □ 

The next result has been announced in [8]. 

Theorem 8.6. Let g > 4 and let N be an irreducible component of N g ^ not 
contained in Ng^k+i- Then: 

(i) for every positive integer k < g — 3, one has coding (N) > k + 2, whereas 
coding (N) = g - 1 ifk = g-2; 

(ii) if N is contained in N g ^, r with r > 3, then codim_4 g (N) > k + 3; 

(iii) if g - 4 > k > g/3, then coding (TV) > k + 3. 
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Proof. By Theorem 7.5 and Remark 7.4, we may assume k < g — 2. By definition, 
there is some irreducible component M of ir^ (N) with dim(M) = dim(iV) + k 
which dominates N via n. We can take a general point (X, Qx,z) € M so that 
C = ©x) is a general point in AT. By Remark 7.4 we may assume X is simple. 

Let R be the unique /c-dimensional component of 7r^ (() containing (X, @x , z) . 
Its general point is of the form £ = (X, Ox , x) with x the general point of the unique 
/c-dimensional component of Sing(©x) containing z, and x has multiplicity r on 
©x- By abusing notation, we may still denote this component by R. Proposition 
6.3 implies that the linear system of quadrics P(Tjy ^) contains all polar quadrics of 
TCt- with £ = (X, O x ,x)e R. 

Thus we have a rational map 

0. (p S -l)r-2 xi? ___> Q (r) 

which sends the general point (6, £) := (&i, . . . , 6 r -2, £) to the polar quadric Qb,£ of 
TCj with respect to bi, . . . , b r —2- It is useful to remark that the quadric Qt.£ is a 
cone with vertex containing the projectivized Zariski tangent space to R at x (see 
Remark 6.4). 

Claim 8.7. [The finiteness property.] For each b € (pa- 1 )' - 2 the map <f> restricted 
to {b} x R has finite fibres. 

Proof of the claim. Suppose the assertion is not true. Then there is an irreducible 
curve Z C R such that, for £ = (X, ®x,x) corresponding to the general point in 
Z, one has Qb^ = Q. Set II = Vert(Q), which is a proper subspace of P 9_1 . 
Consider the Gauss map 

1 = lz : Z-+V3- 1 =V(T x ,o) 

which associates to a smooth point of Z its projectivized tangent space. Then 
Proposition 6.3 implies that 7(£) G R C Vert(Q{) = II. Thus ^f(Z) is degenerate in 
P 9_1 and this yields that X is non-simple, cf. [31]. This is a contradiction which 
proves the claim. □ 

Claim 8.7 implies that the image of the map (f> has dimension at least k, hence 
codim.4 g (iV Si fc) > k + 1. To finish the proof of part (i), we need the following 
information. 

Claim 8.8. [The non-degeneracy property.] The image of the map cf) does not 
contain any line. 

Proof of the claim. Suppose the claim is false. Take a line in the image of the 
map <f>, and let C be the corresponding pencil of quadrics. By Proposition 6.3 
and Remark 6.4, the general quadric in C has rank p < g — k. Then part (i) 
of Segre's Theorem 21.2 in §21 below implies that the Gauss image j(Z) of any 
irreducible component of the curve Z = is degenerate. This again leads to 

a contradiction. □ 

To prove part (ii), wc want to prove that dim(Q^) > k + 1. Remember that 
the image of <fi has dimension at least k by Claim 8.7. If the image has dimenson 
at least k + 1, then by Claim 8.8 it cannot be a projective space and therefore 
dim(Q[ r) ) > jfe + 1. So we can assume that the image has dimension k. Therefore 
each component of the fibre Fq over a general point Q in the image has dimension 
(<?-l)(r-2). 
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Consider now the projection of Fq to R. If the image is positive dimensional 
then there is a curve Z in R such that the image of the Gauss map of Z is contained 
in the vertex of Q. Then X is non-simple, a contradiction (see the proof of Claim 
8.7). 

Therefore the image of Fq on R is constant, equal to a point £, hence Fq = 
(ps-i)r-2 x By Lemma 8 .5 there is a hyperplane H 6 such that TC e = rH s , 
and Q = 2H^. Therefore we have a rational map 

ip:R—* ¥ g - lV 

sending £ to H^. We notice that the image of <j> is then equal to the 2- Veronese 
image if the image of ip. 

By Claim 8.7, the map tp has finite fibres. By an argument as in Claim 8.8, we 
see that the image of tp does not contain a line, hence is not a linear space. Thus 
it spans a space of dimension s > k + 1. Then its 2-Veronese image, which is the 
image of <\> spans a space of dimension at least 2s > 2k + 2 > k + 2. 

To prove part (hi), by part (ii), we can assume r = 2. It suffices to show that 

(2) 

dim(Q^) > k + 1, where := Q\ ■ Suppose instead that dim(Q<;) = k + 1 and 
set £ = 4>(R). We have to distinguish two cases: 

(a) not every quadric in Qq is singular; 

(b) the general quadric in Qq is singular, of rank g — p < g. 

In case (a), consider the discriminant Ac S(, i.e. the scheme of singular quadrics 
in Qq. This is a hypersurface of degree g, which, by Proposition 4.4, contains E 
with multiplicity at least k. Thus deg(S) < g/k < 3 and S contains some line, so 
that we have the corresponding pencil C of singular quadrics. By Claim 8.8, one 
arrives at a contradiction. 

Now we treat case (b). Let g — h be the rank of the general quadric in S. One 
has g — h < g — k, hence k < h. Moreover one has g — h < g — p, i.e. p < h. 

Suppose first p = h, hence p> k. Let s be the dimension of the subspace 

n := p| Vert(Q). 

QeQ MQ)=9-p 

By applying part (hi) of Segre's Theorem 21.2 to a general pencil contained in Q^, 
we deduce that 

(18) 3p < g + 2s + 2. 

Claim 8.9. One has s < r — k. 

Proof. Suppose s > p— k. If II' is a general subspace of n of dimension p — k, then 
its intersection with P(Tr ;2; ), where x € R is a general point, is not empty. Since 
p — k < g — k and X is simple, this is a contradiction (see [31], Lemma 11.12). □ 

By (18) and Claim 8.9 we deduce that p + 2k < g and therefore 3k < g, a 
contradiction. 

Suppose now p < h. Then part (iv) of Segre's Theorem 21.2 yields 

'^<-^ 

The right hand side is an increasing function of p, thus deg(S) < g — h— 1 < 2k— 1 
because g < 3fc < 2k + h. By Lemma 8.4 the locus S contains a line, and we can 
conclude as in the proof of the non-degeneracy property 8.8. 
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□ 

The following corollary was proved independently by Debarre [9] and includes a 
basic result by Mumford [27] . 

Corollary 8.10. Let g > 4. Then: 

(i) every irreducible component S of S g has codimension g + 1 in X g , hence S g 
is locally a complete intersection in X g ; 

(ii) if £ = (X,@x,x) is a general point of S, then either Qx has isolated 
singularities or (X, Qx) is a product of an elliptic curve with a principally 
polarized abelian variety of dimension g — 1; 

(iii) every irreducible component N of -/V Sj o has codimension 1 in A g 

(iv) if (X, Qx) € N is a general element of an irreducible component N of N g fl, 
then Qx has isolated singularities. Moreover for every point x € Sing(6x) ; 
the quadric Q x is smooth and independent of x. 

Proof. Take an irreducible component S of S g and let N be its image via the map 
7T : X g — > A g . Then there is a maximal integer fc < g — 2 such that N is contained 
in N 9t k- Suppose first that 1 < k < g — 3. Then by Theorem 8.6 the codimension 
of N in A g is at least k + 2. This implies that the codimension of S in X g is at least 
g + 2, which is impossible because S g is locally defined in X g by g + 1 equations. 

Then either k — or k = g — 2. In the former case S maps to an irreducible 
component N of N 9i q which is a proper subvariety of codimension 1 in A g . 

If k = g — 2, then by Theorem 7.5 the polarized abelian variety (X, Qx) is a 
product of principally polarized abelian varieties. The resulting abelian variety has 
a vanishing thetanull and so N is contained in the divisor of a modular form (a 
product of thetanulls, cf. e.g. [27], p. 370) which is a component of iV Sj0 . Asser- 
tions (i)-(iii) follow by a dimension count (see Remark 7.4). Part (iv) follows by 
Propositions 4.1 and 8.3. □ 

Finally, we show a basic property of S g . 
Theorem 8.11. The locus S g is reduced. 

Proof. The assertion is well known for g < 3, using the theory of curves. We may 
assume g > 4. 

Let S be an irreducible component of S g , let N = n(S) and k < g—2 the maximal 
integer such that N is contained in N g ^- As in the proof of Corollary 8.10, one has 
either k = Qovk = g — 2. Assume first k — 0, and let £ = (X, Qx, x) £ S be a 
general point. We are going to prove that Sing(9x) is reduced of dimension 0. 

First we prove that x has multiplicity 2 for Qx ■ Suppose this is not the case and 
x has multiplicity r > 3. Since N g fi has codimension 1 all polar quadrics of TCj are 
the same quadric, say Q (see Proposition 6.3 and 8.3). By Lemma 8.5 the tangent 
cone TC^ is a hyperplane H with multiplicity r. Again by Proposition 6.3 all the 
derivatives of Q with respect to points b € H coincide with Q. By Proposition 5.3 
the scheme r\ 2 supported at x is contained in Sing(6x). By taking into account 
Proposition 5.4 and repeating the same argument we see that this subscheme can 
be indefinitely extended to a 1-dimensional subscheme, containing x and contained 
in Sing(6x). This implies that the corresponding component of A^ Sj0 is contained 
in N 9t i, which is not possible since the codimension of iV fli i is at least 3. 

If x has multiplicity 2 the same argument shows that the quadric is smooth. 
By Corollary 3.4, £ is a smooth point of S g and this proves the assertion. 
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Suppose now that k — g — 2. Then by Theorem 7.5 N is contained in the locus 
of products Ag,(i +g -\) and for dimension reasons equal to it and then the result 
follows from a local analysis with theta functions. □ 

Corollary 8.12. If {X, Ox) is a general point in a component of N g $ then @x has 
finitely many double points with the same tangent cone which is a smooth quadric. 

9. A CONJECTURE 

As shown in [8], part (i) of Theorem 8.6 is sharp for k = 1 and 5 = 4 and 5. 
However, we do not expect Theorem 8.6 to be sharp in general. For example, as 
indicated in [8], it is never sharp for k = 1 and g > 6, or for k > 2. In [8] we made the 
following conjecture, which is somehow a natural completion of Andreotti-Mayer's 
viewpoint in [2] on the Schottky problem. 

Recall the Torelli morphism t g : M. g — > A g which maps the isomorphism class of 
a curve C to the isomorphism class of its principally polarized jacobian ( J(C), 0c)- 
As a map of orbifolds it is of degree 2 for g > 3 since the general abelian variety has 
an automorphism group of order 2 and the general curve one of order 1. We denote 
by J g the jacobian locus in A g , i.e., the Zariski closure of t g (M g ) in A g and by TL g 
the hyperelliptic locus in A g , that is, the Zariski closure in A g of t g (H g ), where H g 
is the closed subset of M g consisting of the isomorphism classes of the hyperelliptic 
curves. By Torelli's theorem we have dim( l 7 s ) = 3g — 3 and dim(H g ) = 2g — 1 for 
9> 2. 

Conjecture 9.1. If 1 < k < g — 3 and if N is an irreducible component of N 9y k 
not contained in A® nd ^ z ', then codirn^ ( N) > i^^ 2 ) ■ Moreover, equality holds if 
and only if one of the following happens: 

(i) g = k + 3 and N = H. g ; 

(ii) g = k + 4 and N = J g . 

By work of Beauville [4] and Debarre [11] the conjecture is true for g — 4 and 
g = 5. Debarre [10, 11] gave examples of components of N 9y k for which the bound 
in Conjecture 9.1 for the codimension in A g fails, but they are contained in A^ nd ^ z ', 
since the corresponding abelian varieties are isogenous to products. 

Our main objective in this paper will be to prove the conjecture for k = 1. 

Remark 9.2. The question about the dimension of the Andreotti-Mayer loci is 
related to the one about the loci S g .k introduced in §3. Note that M g ^ = Tr*(N gi k) 
is a subscheme of S g .k- Let TV be an irreducible component of N 9i k not contained 
in N gi k+i, let M be the irreducible component of M g ^ dominating N and Z an 
irreducible component of S 9t k containing M. We now give an heuristic argument. 
Recalling Proposition 4.3, we can consider Z to be well behaved if coding (Z) = 
C^ 1 )- Since M is contained in Z, one has also coding (M) > { k 1 2 1 ) and therefore 
codim.4 s (A) > { kJ ! 2 2 ), which is the first assertion in Conjecture 9.1. In this setting, 
the equality holds if and only if Z is well behaved and M = Z. 

On the other hand, since M is gotten from Z by imposing further restrictions, 
one could expect that M is, in general, strictly contained in Z and therefore that 
coding (A) > ( k + 2 ). 

In this circle of ideas, it is natural to ask if J g [resp. H g ] is dominated by a well 
behaved component of 5 , 9 , 3 -4 [resp. S gig s]. This is clearly the case if g = 4. 
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A related, but not equivalent, question is whether J g [resp. Ti g ] is contained 
in some irreducible subvariety of codimension c < ( 9 2 2 ) [ res P- c < ( S 2 )] m "^f 
whose general point corresponds to a principally polarized abelian variety (A, Qx) 
with Sing(Ox) containing a subschcme isomorphic to Spec(C[ei, . . . , efe]/(ejCj : 1 < 
hj < fc)) with k = g — A [resp. fc = g — 3]. 

One might be tempted to believe that an affirmative answer to the first question 
implies a negative answer to the second. This is not the case. Indeed TL4 is contained 
in the locus of jacobians of curves with an effective even thcta characteristic. In 
this case 64,1 has two well behaved irreducible components of dimension 8, one 
dominating TL4 with fibres of dimension 1, the other one dominating J 4 n #4,0) 
where 6*4,0 is the theta-null locus, see §19 and §10 below and [16]). These two 
components intersect along a 7-dimensional locus in £4 which dominates H4. 

Note that it is not always the case that a component of S 9j k is well behaved. For 
example, there is an irreducible component of <S , g , s _2, the one dominating -4 g ,(i, s _i), 
which is also an irreducible component of S g . 

10. An Example for Genus g = 4 

In this section we illustrate the fact that the quadrics associated to the singular- 
ities of the theta divisor may provide more information than obtained above. The 
locus S4 in the universal family X4 consists of three irreducible components: i) one 
dominating 6*0,4, the locus of abelian varieties with a vanishing theta-null; we call 
it A; ii) one dominating the Jacobian locus J4; we call it B; iii) one dominating 
^4,3+1 , the locus of products of an elliptic curve with a principally polarized abelian 
variety of dimension 3. 

The components A and B have codimension 5 in X4 and they intersect along a 
locus C of codimension 6. The image of C in A4 is the locus of Jacobians with a 
half-canonical 173. The quadric associated to the singular point of order 2 of A in 
Qx has corank 1 at a general point. We refer to [16] for a characterization of the 
intersection 6*0,4 n J4. 

Proposition 10.1. If £ = (A, Qx,x) is a point of C then Sing(6x) contains a 
scheme of length 3 at x. 

Proof. The scheme Sing(9x) contains a scheme Spec(C[e]) at x, say corresponding 
to the tangent vector D\ € kerg x , cf. Section 5. In order that £ is a singular point 
of S4 we must have that the quadrics q and qi associated to x and D\ (cf. equation 
(6) and Proposition 3.3) are linearly dependent. By Proposition 5.3 this implies 
that Sing(Oj( ) contains a scheme of length 3 at a;. □ 

Proposition 10.2. The two components 6*0,4 and J4 of N4.0 in A4 are smooth at 
a non-empty open subset of their intersection and are tangent there. 

Proof. Let £ = (A, ®x,x) be a general point of C so that the singular point x of 
Qx defines a quadric q x of rank 3. Set r/ = (A, Qx) € A4. From the equations 
(3) and (6), one deduces that the tangent cone to A 4 , at 77 is supported on qjr. 
Suppose, as we may by applying a suitable translation, that x is the origin in A. 
Then #0,4 is locally defined by the equation 6*(r, 0) in A4 (see [16], §2). Hence #0,4 
is smooth at r\ with tangent space q x . Now the jacobian locus J4 is also smooth 
at r] by the injectivity of the differential of Torelli's morphism at non-hyperelliptic 
curves (see [15]) and the assertion follows. □ 
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11. The Gauss map and tangencies of theta divisors 

Let (-B, H) be a polarized abelian variety of dimension g, where S is an effective 
divisor on B. As usual we let B = C 9 /A, with A a 2g-dimensional lattice, and 
p : C 9 — > B be the projection. So we have coordinates z = (z\, . . . , z g ) in C 9 and 
therefore on B, and we can keep part of the conventions and the notation used so 
far. Let £ = £(z) be the Riemann theta function whose divisor on C 9 descends to 
S via p. 

If 5 is reduced, then the Gauss map of (B,E) is the morphism 

7 = 7H : S - Sing(S) -> P(T^ ), x -> P(i_ x (T E ,£)), 

where ^-^(Th^) is the tangent space to S at £ translated to the origin. If Sj, = £&(S) 
is the translate of S by the point 6eB defined by the equation £(z — b) = then 
for i£H - Sing(S) the origin is a smooth point for E- x and 7(2;) = P(7h_ x> o). 

As usual we have natural homogeneous coordinates (zi : . . . : in P(Tb ! o) = 
P 9_1 and therefore dual coordinates in the dual projective space P(T V ) = P 9 ~ lV . 
Then the expression of 7 in coordinates equals r ){p{z)) = (di£(z) : ... : d g £(z)) 
with di = d/dzi. 

The following lemma is well known. 

Lemma 11.1. Let (-B,£) be a simple abelian variety of dimension g and S an 
irreducible effective divisor on B. Then the map has finite fibres. Moreover, for 
a smooth point x € S there are only finitely many b € B such that E b is smooth at 
x and tangent to S there. 

Proof. Suppose 73 does not have finite fibres. Then there is an irreducible curve C 
of positive geometric genus contained in the smooth locus of S which is contracted 
by 7h to a point of P 9_1 corresponding to a hypcrplanc II C P 9_1 . Then the 
image of the Gauss map 7c of C lies in II. This implies that C does not generate 
B and contradicts the fact that B is simple. As to the second statement, if it does 
not hold there exists an irreducible curve C such that for all b G C the divisor S(, 
is smooth at x and tangent to S. Then the curve C — {x — b : b e C} is contained 
in S and contracted by 72 ■ This contradicts the fact that has finite fibres as we 
just proved. □ 

Definition 11.2. Let h be a natural number with 1 < h < g — 1. The subscheme 
T h (B, E) of B x B h is defined in C 9 x (C 9 ) h with coordinates (z, u u . . . , u h ) by the 
equations 



£(z)=0, £(*-«i) = 0, ... ,t(z-u h )=0, 

(19) 



rk 



di£(z -m) ■■■ d g i(z - m 



< h. 



\di£{z-u h ) ■■■ d g £(z-u h )J 



The projection to the first factor induces a morphism p\ : Th(B, S) — > 5. Note 
that for z = 1, . . . , h the variety Ei of codimension h+1 < g in B x B h defined by 
the equations 

£(z) = £(2 — Uj) = j and Uj = 0, 
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is contained in T?i(i?,S). Moreover, the expected codimcnsion of the irreducible 
components of Th(B, H) is g + 1. We will say that an irreducible component T of 
T/j(_B,S) is regular if: 

(i) for i = l,...,h; 

(ii) on a non-empty open subset of T all the rows of the matrix in (19) are 
non-zero. 

In particular, if T is regular then pi(T) % Sing(S). 

Proposition 11.3. If B is simple each regular component ofTh(B,E) has dimen- 
sion hg — 1 . 

Proof. Let us first assume h = 1 and denote a component of 7/,(i?, E) by T. By 
composing pi with the Gauss map 7 = 73, we obtain a rational map <f> : T --■> 
P(T^ ). We shall prove that ^ has finite fibres. Let v be a point in the image of 
<j> coming from a point in the open subset as in Definition 11.2, (ii). By Lemma 
11.1 there are only finitely many smooth points 21, . . . , z a <G S such that 7(2^ = v 
for i = 1 , . . . , a. For each 1 < i < a we consider the theta divisor defined by 
£,(zi~- u) = 0. Again by Lemma 11.1 there are only finitely many points un, . . . , uu t 
in it such that (zi,Uij) may give rise to a point on T. So <j> has finite fibres. Thus 
it is dominant and T has dimension g — 1. 

Now we prove the assertion by induction on h. Consider the projection q : T — > 
B x B 11 ^ 1 by forgetting the last factor B. If the image T" of T is contained in 
T^_i(B,5), then it is contained in a regular component of T/j_i(B,S), hence by 
induction the codimension of T is at least g + 1 , while we know from the equations 
that it is at most g + 1, and the assertion follows. 

Suppose T' is not contained in Th-i(B, S). Let U(ft— 1, g—1) — ► G(ft— 1, <?— 1) be 
the universal bundle over the Grassmannian of (/i— l)-planes in P 9_1 . Then we de- 
fine a rational map ip : T --■> \J(h— 1, g— 1) in the following way. If (z, ui, . . . , Uh) is 
a general point in T, then by the assumption that T" 2 Th-i(B, S) the hyperplanes 
7(2), 7(2 — ui), . . . , 7(2 — Uh-i) arc linearly independent and we define 

i>(z,uu ...,u h ) = ((7(2), 7(2 - ui), . . .,7(2 - u h _i)), 7 (z - u h )). 

We claim that Lemma 11.1 implies that the general fibre of ip has dimension < h(h— 
1). Indeed, in a fibre of ip the points 7(2), 7(2—^1), . . . , j(z— Uh—i) vary in a (h— 1)- 
dimcnsional space giving at most h{h—\) parameters and if 7(2), 7(2—1*1), . . . , 7(2— 
Uh) are fixed there are for (2, ui, . . . , Uh) only finitely many possibilities. Thus 
dim(T) is bounded from above by 



dim(U(ft- l,g- 1)) + h{h- 1) = h{g - h) + (h - l) + h{h-l) = hg - 1, 



We will also consider the closed subscheme T®(B, 5) of Th(B, 5) which is defined 
in C 9 x (C 9 )' 1 by the equations 

£(2) = 0, £(2-O=0, i = l,...,fc 



and this proves the assertion. 



□ 



and 



rk 



/ 9^{z) 
di£{z - ui) 



d g £{z - m) 



= 0. 



\d^(z - u h ) 



d g £(z - u h )J 



22 



CIRO CILIBERTO AND GERARD VAN DER GEER 



Finally we will consider the closed subset 7h(X, B) which is the union of T°(X, B) 
and of all regular components of Th(B, E). Look at the projection 

p = p 2 :T(X,B)^B h . 

Definition 11.4. We define No.h(B,E) to be the image of Th(X,B) under the 
map p. More generally, for each integer k we define 

N Kh (B,E) := {u = (u u ...,u h ) G B h : u u . . . , u h ^ 0, dim^ 1 ^)) > fc}. 

Roughly speaking, N k ^h{B,E) is the closure of the set of all (ui, . . . ,Uh) € B h 
such that E contains an irreducible subvariety V of dimension n > fc and for all 
z G V either: 

(a) z, z — ui, . . . ,z—Uh are smooth points of S and j(z), 7(2 — tii), ... , 7(2 — Uh) 
are linearly dependent, or 

(b) all the points z, z — u\, . . . , z — Uh are contained in Sing(S). 

In case (a) we say that the divisors S, S Ul , . . . , Sh h , all passing through z with 
multiplicity one, are tangentially degenerate at z. 

In case h = 1 we have only two divisors which are just tangent at z. We will 
drop the index h if h = 1. Thus, Nq(B, S) is the set of all u such that S and S u 
are either tangent, or both singular, somewhere. Note that, if S is symmetric, i.e., 
if £(2) is even, as happens for the Riemann theta function, then N (B, S) contains 
the divisorial component 2S := {2£: £ e 5}, since 7(£) = 7(— £) for all smooth 
points £ e S. 

One has the following result by Mumford (sec [27], Proposition 3.2). 

Theorem 11.5. If (B, 5) is a principally polarized abelian variety of dimension g 
with S smooth, then N (B, H) is a divisor on B algebraically equivalent to 5. 

As we will see later, the following result is related to Conjecture 9.1. 

Proposition 11.6. Suppose that (-B,S) is a simple principally polarized abelian 
variety of dimension g. Assume that (B,E) ^ N g _ k . Then for k > anrf 1 < /i < 
<7 — 1 and for every irreducible component Z of N k ^ h (B,E) one has codim B h(Z) > 
k + l. 

Proof. By the definition of N^ ^B, 5) and by the fact that Sing(S) has dimension 
< fc an irreducible component of N k ^ h (B,E) can only be contained in the image of 
a regular component of Th(B,E). The assertion follows by Proposition 11.3 and 
the fact that the fibres of p 2 on a regular component have dimension > fc. □ 

12. Properties of the loci N kJl (B,E) 

We will prove now a more precise result in the spirit of Proposition 11.6. 

Proposition 12.1. Suppose that (B,E) is a simple, principally polarized abelian 
variety of dimension g. Assume that N kt h(B,E) is positive dimensional for some 
h>l and fc > 1. Then (B,E) e A^fc-i- ' 

Proof. We may assume (B, E) £ N 9yk , otherwise there is nothing to prove. We may 
also assume ft > 1 is minimal under the hypothesis that N kt h(B,E) has positive 
dimension. 

Let (ui, • • • , Uh) be a point in N k ^(B,E), so that So := 5 and Sj := E Ui ,i 
I // . are tangentially degenerate along a non-empty subset V a of an irreducible 



ANDREOTTI-MAYER LOCI AND THE SCHOTTKY PROBLEM 



23 



sub variety V of B of dimension k such that Ei for i = 0, . . . , h are smooth at all 
points »eV . 

For j = 0, . . . , h the element := di£(z — Uj) (with the convention that uq = 0) 
is a section of 0(Ej) when restricted to V since Ej contains V. We know that for 

given j not all are identically zero on V°. Our assumptions on the tangential 
degenerateness and minimality tell us that there exist non-zero rational functions 
Oj such that we have a non-trivial relation 

h 

(20) J2 a i s ? =0 for * = 1 , ••-,£?■ 

Suppose that the aj are regular on all of a dcsingularization / : W — > V of V. 
Then they are constant and the relation a j s ^ — holds on the whole of W. By 
writing relation (20) in different patches which trivialize the involved line bundles, 
and comparing them, we see that, if the transition functions are not all proportional, 
then we can shorten the original relation by subtracting two of them. This would 
contradict the minimality assumption. Therefore, we have that all of the divisors 
/* (S — E Uj )|y with j = 1, . . . h are linearly equivalent. Since u = (ui, . . . , Uh) varies 
in a subvariety of positive dimension this implies that the map Pic°(i3) — > Pic(Vl^) 
has a positive dimensional kernel, and this is impossible since B is simple. So there 
exists an index j such that the function aj has poles on a divisor Zj of W. Note 
now that a point which is non-singular for all the divisors Ej, j — l,...,h, is 
certainly not a pole for the functions aj. Therefore, for each j = 1, . . . , h, there 
an I depending of j such that Zj is contained in the divisorial part of the scheme 
/*(Sing(S^)). Moreover, since Zj moves in a linear system on W, it cannot be 
contracted by the birational morphism /. This proves that Eg is singular along a 
variety of dimension k — 1 contained in V, which proves the assertion. □ 

Corollary 12.2. Suppose that (B, S) is a simple, principally polarized abelian va- 
riety of dimension g. Assume that (-B,S) ^ ^g,o, that is, S is smooth. Then every 
irreducible component Z of Nq^(B, S) is a divisor of B h . 

Proof. Each irreducible component Z of N ^{B, S) is dominated by a regular com- 
ponent T of Th{B,E), which has dimension hg — 1 by Proposition 11.3. The map 
p : T — > Z is finite by Corollary 11.1. The assertion follows. □ 

Remark 12.3. If we have two divisors So, Si which are tangentially degenerate 
along an irreducible fc-dimensional variety V whose general point is smooth for 
both S , Si, then So, Si are both singular along some (k — l)-dimensional variety 
contained in V. This can be easily proved by looking at the relation (20) in this 
case, and noting that the polar divisors Zj is contained in /*(Sing(Sj)), j = 0, 1. 

Remark 12.4. Suppose (B,E) £ N g fi- Then Nq(B,E) is described by all differ- 
ences of pairs of points of S having the same Gauss image. 

Suppose (B,E) € N g _o — N g ^ and assume {x, — x} = Sing(S) have multiplicity 
2 and the quadric Q x = Q- x is smooth. It may or may not be the case that 
b = —2x e N\(B,E). In any case, we claim that Ni(B,E) — {— 2x} is contained 
in the set of all differences of points in J~ 1 (Q X ) with x. Let us give a sketch of 
this assertion, which provides, in this different argument for the proof of 

Proposition 11.6. 
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If b £ N\(B,E) — {— 2x}, there is a curve C C S such that for t £ C general 
7=(i) = Js(t + b). Along C the divisors S and Sf, are tangent, hence the curve 
contains x (see Remark 12.3). Note that the curve C is smooth at x. Indeed, 
locally at x, the divisor 5 is a quadric cone of rank g — 1 in C 9 with vertex x, 
whereas is a hyperplane through x, and they can only be tangent along a line. 

Thus it makes sense to consider the image of x for 7c, which is a point on Q x . 
The point x + b£C + bcZis smooth for 5 and 7(2; + b) is clearly tangent to Q x 
at 7c (x). 

13. On N g -2 AND iVg_ 3 FOR Jacobians 

The following result shows that the bound in Proposition 11.6 is sharp in the 
case h = 1. Recall that a curve C is called bielliptic if it is a double cover of an 
elliptic curve. 

Proposition 13.1. Let C be a smooth, ireducible projective curve of genus g and 
let (J = J(C), 9c) be its principally polarized jacobian. Then one has: 

(i) {O c (p - q) € J : p, q € C} C 7V S _ 3 (J, 6 C ); 

(ii) either C is bielliptic or the equality holds in (i); 
(hi) if C is hyper elliptic, then 

{O c (p-q)eJ:p,qe C, h°(C, O c ( P + q)) = 2} = N g _ 2 (J, Q c ). 

Proof. We begin with (i). We assume that C is not hyperelliptic; the hyperelliptic 
case is similar and can be left to the reader. We may identify C with its canonical 
image in P 9_1 . Moreover, we identify J with Pic 9_1 (C) and 6 C C Pic 9_1 (C) with 
the set of effective divisor classes of degree g — 1. Then the Gauss map 7c := 7e c 
can be geometrically described as the map which sends the class of an effective 
divisor D of degree g — 1 such that h°(C,Oc(D)) = 1 to the hyperplane in P 9 ^ 1 
spanned by D (see [15], p. 360). 

Take two distinct points p\,P2 on C. Then \u>c(— Pi — P2)\ is a linear series of 
degree 2g — 4 and dimension g — 3. For i = 1, 2 we let Vi be the subvariety of 6c 
which is the Zariski closure of the set of all divisor classes of the type E+Pi, where 
E is a divisor of degree g — 2 contained in some divisor of \u>c(~ Pi — P2 ) | - Clearly 
dim(Vi) = g — 3, hence Vi is not contained in Sing(Oc) which is of dimension g — 4. 
If u denotes the divisor class P2—P1 then x 1— » x + u defines an isomorphism V\ = Vi- 
Moreover, for x in an non-empty open subset of V\ we have 7c (z) = jc(x + u). 
This proves (i). 

Conversely, assume there is a point u <G Pic°(C) — {0}, and a pair of irreducible 
subvarieties V, V of Oc of dimension .9 — 3 such that x x + u gives a birational 
map from V\ to Vi- For 1 in a non-empty open subset U C Vi we have 7c (a;) = 
7c(a; + u )- If -D and D' are the effective divisors of degree g— 1 on C corresponding 
to x and x + u and if 1? is the least common divisor of D and D' then by the 
geometric interpretation of the Gauss map 7c there is an effective divisor F with 
deg(£) = deg(F) such that D + D' - E + F = K C - Thus (2D - E) + F = K c - u. 

Consider the linear series \Kq — u\, which is a g^g-2- ^ this linear series has 
a base point q, then there is a point p such that Kq — u — q = Kq — p, i.e. 
D' — D = u = p — q, proving (ii). So we may assume that \Kc — u\ has no base 
point. If C is not bielliptic, then \Kc — u\ determines a birational map of C to 
a curve in P 9 ~ 2 . On the other hand it contains the (g — 3) -dimensional family of 
divisors of the form (2D — E) + F, which are singular along the divisor D~E. This 
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is only possible if deg(D — E) = 1, i.e., u = D' — D = p — q, with p,geC. But in 
this case \Kc — u\ has the base point q, a contradiction. This proves (ii). 

Assume now C is hyperelliptic. Let p\ + p 2 be an effective divisor of the g\ on C 
with pi 7^ p 2 ■ Then \ujq {— Pi ~T>2)\ is a linear series of degree 2g — 4 and dimension 
g — 2. For t = 1,2 we let Vj be the subvariety of Oc which is the Zariski closure of 
the set of all divisors classes of the type E+p i} where E is a divisor of degree g — 2 
contained in a divisor of \u)c(— Pi — P2)\- The variety Vi has dimension g — 2 and 
is not contained in Sing(6c), which is of dimension g — 3. The translation over u 
induces an isomorphism V\ = Vi and for i in a non-empty subset U of V\ we have 
lc{x) — 7c(a; + u). Hence the left hand side in (hi) is contained in iV s _2(J, 6c)- 

Finally, assume there is a point u G Pic°(C) — {0}, and a pair of irreducible 
subvarieties V, V of Oc of dimension .9 — 2 such that translation by u gives a 
rational map V --■> V' with 7c(#) = 7c( x + w) on a non-empty open subset of V. 
Let D, D' be the effective divisors of degree g — 1 on C corresponding to x and 
x + u. As in the proof of part (ii), let E be the least common divisor of D and D' . 
In the present situation the linear scries \Kq — u\ of dimension g — 2 contains the 
(g — 2) dimensional family of divisors of the form 2D — E, which are singular along 
the divisor D — E. This means that 2{D — E) is in the base locus of \Kc~u\. This is 
only possible if D = E + q for some point q £ C, and Kq — u — 2q= (g — 2)(p + q), 
where p is conjugated to q under the hyperelliptic involution. In conclusion, we 
have u = p — q and the equality in (iii) follows. □ 

Remark 13.2. The hypothesis that C is not biclliptic is essential in (ii) of Propo- 
sition 13.1. Let in fact C be a non- hyperelliptic bielliptic curve which is canon- 
ically embedded in P 9 " 1 . Let / : C — > E be the bielliptic covering. One has 
f*O c ^O e ® C V , with £® 2 ~ O e (B), where B is the branch divisor of /. 

Let u € Pic (E) — {0} be a general point, which we can consider as a non- 
trivial element in Pic°(C) via the inclusion /* : Pic°(£;) -» Pic°(C) Note that / is 
ramified, hence /* is injective. We want to show that u G N g -z{J, 9c), proving 
that equality does not hold in (i) in this case. 

The canonical image of C is contained in a cone X with vertex a point v G P ff_1 
over the curve E embedded in P^~ 2 as a curve of degree g — 1 via the linear system 
|£|. Let us consider the subvariety W of |£| consisting of all divisors M G |£| such 
that there is a subdivisor p + q of M with p,q e E and p — q = u. It is easily seen 
that PF is irreducible of dimension — 3. 

Notice that for M general in W one has M = p + q + N, with N effective of 
degree g - 3. Therefore we may write ivT c = f*(M) = f*(p) + f*(q) + F + F' , 
where F,F' are disjoint, effective divisors of degree g — 3 which are exchanged by 
the bielliptic involution. 

We let V be the (g — 3)-dimensional subvariety of 6c described by all classes 
of divisors D of degree g — 1 on C of the form D = f*(p) + F, as M varies in 
W. Then D + u = D' := /*(<?) + F and D and £>' span the same hyperplane 
through v in P s_1 . Therefore, if x G V is the point corresponding to D, one has 
7c 0*0 = 7c( a; + M )- This proves that u G N g ^ 3 (J, 0c)- 

14. A BOUNDARY VERSION OF THE CONJECTURE 

We will now formulate a conjecture. As we will see later, it can be considered 
as a boundary version of Conjecture 9.1 (see also Proposition 11.6). 
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Conjecture 14.1. Suppose that (-B,S) is a simple principally polarized abelian 
variety of dimension g. Assume that (P,S) ^ N g ^ for all i > k > 1. TTien i/iere 
is an irreducible component Z of Nk{B, S) wif/i codime(^) = fc + 1 if and on/y i/ 
one o/ f/ie following happens: 

(i) either g > 2, k = g — 2 and B is a hyperelliptic Jacobian, 

(ii) or g > 3, k = g — 3 and B is a Jacobian. 

One implication in this conjecture holds by Proposition 13.1. Note that the 
conjecture would give an answer for simple abelian varieties to the Schottky problem 
that asks for a characterization of Jacobian varieties among all principally polarized 
abelian varieties. For related interesting questions, see [30]. 

15. Semi-abelian Varieties of Torus Rank One 

Let (.B, S) be a principally polarized abelian variety of dimension g — 1. The 
polarization S gives rise to the isomorphism 

4>b:B^B = Pic°(P), 6 -> O s (S - S 6 ). 

and we shall identify B and P via this isomorphism. Thus an element o G P = P 
determines a line bundle L = Lb = Ob (S — S(,) with trivial first Chern class. We 
can associate to L a semi-abelian variety X = Xs.b, namely the G m -bundle over 
B defined by L which is an algebraic group since it coincides with the theta group 
Qb '■= Q{L) of L (cf. [25], p. 221). This gives the well-known equivalence between 
B and Ext(P, G m ), the group of extension classes of B with G m in the category of 
algebraic groups (see [34], p. 184). 

Both the line bundle L and the G m -bundle X determine a P 1 -bundle P = P(L ® 
Ob) over B with projection 7r : P — > B and two sections over B, say so and Soo 
given by the projections L Ob — » £ and L Ob — > Ob- If we set Po = sq(B) 
and Poo = Soo(P) then we can identify P — Po — Poo with X. By [17], Proposition 
2.6 on page 371 we have Op(l) = Op(Po). We can complete X by considering the 
non-normal variety X = Xs.b obtained by glueing Po and Poo by a translation 
over b e B = B. On P we have the linear equivalence p> — Poo = 7r _1 (S — St). 
We set P := Poo + tt^ 1 (S) and put M r = Op(E). This line bundle restricts to 
C?b(S) on Po and to OB(2b) on P^, and thus descends to a line bundle M — 
on W We have 7r«(0 P (P)) = O s (S) Ob(S 6 ) and P°(P,M P ) is generated by 
two sections with divisors Poo + 7r _1 (S) and Po + 7r _1 (Sft). One concludes that 
H°(X,M) corresponds to the sections of Mp such that translation over 6 carries 
its restriction to P to the restriction to Poo. It follows that h°(X,M) = 1 with 
effective divisor S, which is called the generalized theta divisor on X. 

Analytically we can describe a section of Ojf(S) on the universal cover C x C 9_1 
by a function 

£(t,z) +u£(t,z - uj), 

where u e C 9 " 1 represents b e B = C 9 " 1 /^ -1 + tZ 9 ' 1 , £(t,z) is Riemann's 
theta function for B and u = exp(27r«C) is the coordinate on C*. This is called the 
generalized theta function of X. 

Let D be the Weil divisor on X that is the image of Po (or, that is the same, of 
Poo)- We consider a locally free subshcaf T vcrt of the tangent sheaf to X, namely 
the dual of the sheaf fi 1 (logP) of rank g. If d e D is a point such that on the 
normalization P near the two preimages zi, ... , z g -\,u and zi+b,..., z g -i+b, v are 
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local coordinates such that u = defines Po (resp. v = defines Poo) with uv = 1 
then Tvort is generated by d/dzi, . . . , d/dz g -i, ud/du— vd/dv. Here z\, . . . , z g -\ arc 
coordinates on B. We interpret local sections of T vert as derivations. In particular, 
if an effective Cartier divisor Y of X has local equation / = 0, then for each local 
section d of T vert , the restriction to Y of df is a local section of Oy(y)/Oy. Then 
the subscheme Sing vcrt (S) of 5 is locally defined by the g equations 

(21) dj = modulo / in O x (S)/O x 

with / = a local equation of S and di local generators of T vert . 
The equations for Sing vcrt (S) on X are thus given by 

£(t,z) + u£(t,z-lj) =0, 
£(t,z- w) =0, 
d£(T, z) + ud£(r, z-u)=0, {l<i<g- 1). 



The points in Sing vort (S) are of two sorts depending on whether they lie on the 
double locus D of X or not. The singular points of Sing(S) on X = X — D are 
the points (z,u), with ii / 0, which are zeros of £( T i z ) and £( t j z — u>) and such 
that 7(r, z) = —uj(t,z — w). That is, geometrically, these correspond under the 
projection on B to the points on B where H and Sf, are tangent to each other. To 
describe the singular points of Sing(S) on D, we consider the composition 

0: B = P -> P-%X, 

where v is the normalisation. Then we have (/)~ 1 (Sing vcrt (S)) = Sing(S) and the 
same if we identify B with . 

Points of Sing vert (S) determine again quadrics in Pf" 1 as follows. Note that the 
projective space P(Tx,o) contains a point Pb corresponding to the tangent space 
T<a m C Tx of the algebraic torus G m at the origin. Recall that we write 7(7-, z) for 
the row vector 

7(7-, z) = (d 1 ^,...,d g - 1 ^)(T,z). 

Then a singular point determines a, possibly indeterminate, quadric defined by the 
matrix 

(22) ( ° J{r,z-u)\ 

with M the (g — 1) x (g — 1) matrix (d/dTij^(r 7 z) + ud/dTij^(r 7 z — u>)). Note that 
we have 7(7", z) = —1*7(7, z — ui). The quadric passes through the point Pb- For a 
point on D the quadric is a cone with vertex Pb over a quadric in ps -2 given by M. 

Remark 15.1. The above considerations show that a point in Sing vert (S) has to 
be regarded as a point of multiplicity larger than 2 if the matrix (22) vanishes 
identically. This can happen only if z and z + lo are both singular for S. 

16. Standard Compactifications of Semi-Abelian Varieties 

Let (B, S) be a principally polarized abelian variety. We assume now that 
dim(B) = g — r with r > 1 and extend the considerations of the previous sec- 
tion. 
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The extensions of B by are parametrized by Ext 1 (2?, G^) — B r '. To a point 
(61, . . . ,b r ) G B r one associates the GJ^-extension X = Xb obtained as the fibre 
product of theta groups Q bl Xg • • • x b Gb r ■ 

One of the type of degenerations of abelian varieties that we shall encounter 
are special compactifications of semi-abelian varieties. We shall call them standard 
compactihcations of torus rank r. Let b = (b\, . . . , b r ) G B r . The algebraic group 
X = Xb sits in a P 1 Xg • • • x B P 1 -bundlc n : P — > B that is obtained as the fibre 
product over B of the P^bundles P bz = ¥(L bi @O b )- The complement P - X is a 
union of 2r divisors £T =1 + , where (resp. n^) is given by taking 
(resp. 00) in the i-th fibre coordinate, with projections ^^,^1,00 to B. 

We now define a non-normal variety obtained from P by glueing Hq^ with IIoo 
for i = 1, . . . , r. This identification depends on a r x r-matrix T = (iy) with entries 

from G m such that tu — 1 and tij = tj*. Let : B — » (resp. Soo) be the 
zero-section (infinity section) of P bi . We glue the point 

{/3, xx, . . . ,x l - ll s ( l \p),x i+1 , . ..,x r ) 

(i) 

on Ilg with the point 

(J3 + bi,t it ixi, . . . ,ti t i-iXi-i,sQ(/3),ti >i+ iXi + i, . . . ,t i>r x r ) 

on IIto ■ We denote the resulting variety by X. It depends on the parameters b G B r 
and t e Mat(r x r, G m ). 

We have the linear equivalences IIq^ — 11^ = it* (S — S(, 4 ) . We set = IIoo + 
tt*(S) = E n S + and s * = n °° - nS and M := Mr = O v (E). This line 

bundle restricts to w (£; + < ( s )) on n o } and to + <<x>( s f> i )) on 

' Hoc ' 

Thus, by the definition of the glueing, M descends to a line bundle M := on 
X. We have 

7r*(M) =( ®r =1 (O b © L^ 1 )) ® O b (S) 

— ©fc=l ( ffi0<»i<...<i fc <r C , B(S 6n+ ...+b s j). 

Hence we have h°(F,M) = 2 r . As in the preceding section one sees that only a 
1-dimensional space of sections descends to sections of M on I. In terms of coor- 
dinates . . . , £ r , zi, . . . , Zg-r) on the universal cover of X, where (zi, . . . , z g - r ) G 
C 9 ~ r are coordinates on the universal cover of B, a non-zero section of M is given 

by 

^ uiti£(r, z - uji), 

7C{l,...,r} 

where I runs through the subsets of {1, . . . , r}, uj = Yl i£l Ui with m — exp(27r£i), 
ti = n»,jej,»<j bi = J2iei^i an( i w i e C 9 ~ r represents 6/ G P. This is the 
generalized theta function of X, whose zero locus is the generalized theta divisor S 
of X. 

Next we look at the singular points of S. All points in S n P are singular points 
of S. However, just as in the rank one case in the preceding section we will in 
general disregard these singularities of S, and we will only look at the so called 
vertical singularities, which we are going to define now (cf. [27], §2). 

The locally free subshcaf T VO rt of rank g of the tangent sheaf T-^ is the dual 
of Sl^logP). Its pull back to P is generated, in the (u, ^-coordinates, by the 
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differential operators Uid/dui — Vid/dvi with UiVi — 1 for i = 1, . . . ,r and dj — 
d/dzj with j = 1, . . . ,g — r. We interpret local sections of T VO rt as derivations 
as above and define the scheme Sing vert (S) of vertical singular points of S as the 
subscheme of S defined by the equations (21) with / = a local equation of S for 
all local sections d £ T vort . This is independent of the choice of a local equation. 

Lemma 16.1. Let (X,E) be a standard compactification of a semi-abelian variety 
X of torus rank r with abelian part (B,E). If dim(Sing vort (S)) > 1 then (B,E) € 

Ng- r fi. 

Proof. The compactification A" is a stratified space and the (closed) strata are 
(standard) compactifications of semi-abelian extensions of (B,E) of torus rank s 
with < s < r. In view of the relations ujtjdi^(z — u>i) = the vertical 
singularities of E correspond to points where 2 s translates of 5 are tangcntially 
degenerate. Therefore we have dim(Ni t h(B, S)) > 1 with h = 2 s . By Lemma 12.1 
it follows that (B, E) e N g - rfi . □ 

17. Semi-abelian varieties of torus rank two 

In the compactification of the moduli space of principally polarized abelian vari- 
eties of dimension g we shall encounter two types of degenerations of torus rank 2. 
The first of these is a standard compactification introduced above and its normal- 
ization is a P 1 x P 1 -bundle over a principally polarized abelian variety of dimension 
g — 2. For such a standard compactification the equations for Sing vert (S) are given 
in terms of the (u, z)-coordinates by the system (we write t instead of ti i2 ; note 
that t ^ 0) 

£(z) - tuiu 2 £,(z — lji — uj 2 ) =0, 
ui£(z - lui) + tuiu 2 i{z — ui — uj 2 ) =0, 
u 2 £(z - lu 2 ) + tuiu 2 ^{z - LUl - UJ 2 ) =0, 
dii{z) + uidi£(z — oji) + u 2 di^(z - u 2 ) + tuiu 2 d^(z — u>\ - u 2 ) =0, 

* = 1 ff-2. 

From this and the analogous equations in the ^-coordinates (with mvi = 1) we see 
that the vertical singular points of 5 are essentially of three types: 

(i) A point x G D that is the image via ip : X — > X of a point in II^o n n 2i o — B, 
(i.e. in the (u, z) coordinates one has Ui = u 2 = 0) is a vertical singular point of S 
if and only if it corresponds to a singular point of S on IIi.o H IT2.0 — B and to a 
singular point of E bl+b2 on IIi j00 n U 2 oo = B. 

(ii) A point x G D which is the image via ip of a point in Ilj^o but not of a point 
in Yls-jfi, (i.e., in the (u, z) coordinates one has Ui = 0, 7^ 0, for a j = 1, 2) is 
a vertical singular point of S if and only if 

£(t,z) = 0, C(t,z-lo 3 ^)=0, 
di€(r,z) + u 3 -jdi£(T,z - u 3 -j) =0, i = 1, ... ,g - 2. 

i.e. if and only if z and z — b are in S and js(z) — Je(z — b). 

(iii) A point x ^ D, (i.e. in the (u,z) coordinates one has mi ^ ^ u 2 ) is a 
vertical singularity if and only if z is a singular point of the divisor H £ \2Eb 1 +b 2 \ 
defined by the equation 

£(t, z - Wi)£(t, z-u) 2 )=t f (t, z)£(t, z-l)!-u) 2 ). 
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By the way, this occurs even in case (ii) above. Note also that, by the above 
equations, the existence of a vertical singularity implies that the theta divisors 
5, S&j, S& 2 and S& 1+ & 2 are tangentially degenerate at some point x of B, i.e., 
zeiVo, 3 (B,S). 

We call of type (i) , (ii) or (iii) the singular points of S according to whether cases 
(i), (ii) or case (iii) occur. 

Remark 17.1. A point x in Sing vert (S) again determines a quadric Q x in P 9_1 . 
It is useful to remark that: 

(a) in case (i) the quadric Q x is a cone with vertex the line := {Pb 1 ,Pb 2 ) 
given by the tangent space to the toric part over the quadric Q z in ps~ 3 which 
corresponds to the singular point z of S; 

(b) in case (ii), say we are at a point with u\ = 0, u 2 ^ 0. Then Q x is a cone 
with vertex P& 1 over the quadric in the hyperplane u\ = with matrix 

f -7(t,*)*\ 
\-l(r,z) M ) 

with 7(t, z) = (<9i£, . . . , <9 9 _ 2 £;)(t, z ) an d the matrix M is given by 

M = (d/dTij£(T, z) + ud/dTij£(r, z - u))i<i,j< 9 -2- 

In §15 we saw that all rank 1 compactifications of G m -extensions of a principally 
polarized abelian variety B form a compact family B. This is no longer the case 
in the higher rank case. This is where semi-abelian varieties of non-standard type 
come into the picture. This will depend of choices. It is good to see this in some 
detail in the rank 2 case. 

Given a principally polarized abelian variety (B, S) of dimension g — 2, all stan- 
dard rank 2 compactifications of (-B,S) are of the form (X,S) with X = Xs,b,t 
with b = (61,62) € B x B and teC*. Thus the parameter space may be identi- 
fied (up to dividing by automorphisms) with the total space of the Poincare bundle 
V — > B x B minus the 0-section P$ . It is then natural to compactify this by looking 
at the associated P 1 -bundle and by glueing on it the 0-section P with the infinity 
section P^. This in fact works and it is explained in [26], §7, and in [29]. We 
describe next the new objects that arise. 

We denote by Li the line bundle associated to hi, for i = 1, 2. We consider again 
the P 1 x P 1 bundle P on B as in §16 and in the glueing operation described in §16, 
we let t = t\2 tend to (or cquivalently to 00). Letting t — > 00, one contracts Ili o 
and n 2 ,o to the section A — ITi i(X) n n 2 ,oo — B, and Ili j00 and n 2j0 o to the section 
A = II^o n n 2 ,o — B. In order to properly describe the glueing process, we have 
first to blow-up the two sections A and A in P. Let us do that. Let w : P — > P be 
the blow-up, on which we have the following divisors: a is the exceptional divisor 
over A and 8 is the exceptional divisor over A; (3, 7, e, ( are the proper transforms 
on P of 11^00,112,0,11^0,112,00, respectively. 

We will abuse notation and denote by the same letters the restrictions of these 
divisors on the general fibre $ of P over B, which is a P 1 x P 1 blown-up at two 
points, hence a P 2 blown-up at three points. Note that a, (3, 7, 5, e, ( are P 1 -bundles 
over B and one has 

ol = ¥(LY © L%), 7-P(L 2 ®O b ), e^P(L!©0 B ), 
[ > <5 = P(Iiffii 2 ), C = P((£i ®L 2 )®L 2 ), |8SP((Li®L2)eii). 
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At this point one could be tempted to suitably glue a with 7 and e and S with 
(3 and (. This however, as (23) shows, does not work. The right construction is 
instead the following. 

One considers two P 2 -bundles fa : P\ — > B, i = 1,2, associated to the vector 
bundles L\ L 2 and L\ ® L 2 on B, i.e., 

P\ = F{L\ ®L 2 V ® O b ), P\ = P(ii @L 2 @ O b ). 

There are three relevant P^subbundles of the bundles Pf , i = 1,2, namely 



y_ = V(LY®L%), 7 = P(L^O B ), e_=¥{LX®0 B ) in P{ 
S = F(L 1 ®L 2 ), C = F{L 1 ®O b ), P = P(L 2 ®O b ) in P\. 



As (23) and (24) show, we can glue P with P\ and Pj in such a way that a and (5 
are respectively glued to a and 5; e is glued to e and /3 to /? with a shift by —61, 
and £ to £ and 7 to 7 with a shift by —b 2 . The resulting variety is denoted by 
X = Xs,b- As usual, we will denote by D its singular locus. On P we have the 
line bundle 

M - w*Op(U hoo + n 2jCO ) ® Op(S - a), 

where we write L instead of w*(tt*(L)) for a line bundle, or divisor, L on B. With 
similar notation, one has 

M-Op( a + /3 + C + S) = Op(5 + e + C + S bl ) 

= Op(5 + 7 + f3 + E b2 ). 

One has O p j(l) = O p t(a) and the following linear equivalences 

(26) a-^ = L\, a-l=L\ 

where again we write L instead of <j>*(L), i = 1,2, for a line bundle, or divisor, L 
on £> (see again [17], Proposition 2.6, p. 371). From (25) and (26) one deduces 
that M glues with the line bundle M\ = O p » (a + S) and with the line bundle 

M.\ — Optt(S), to give a line bundle M on X. In the evident coordinates (u, z) = 

((ui, u 2 ), (zi, . . . , z g - 2 )), which can be considered as coordinates on P— (aU- • -U(), 
the sections of M can be expressed as 

o^(r, z) + 01M1 ^(r, 2 - wi) + a 2 u 2 ^(r, z - lu 2 ) 

with a, ai, a 2 complex numbers. By taking into account the glueing conditions, one 
sees that only a one-dimensional subspace V of H°(P, M) gives rise to a space of 
sections of H°(X, M); V is generated by 

(27) £(t, z) + Ui£(t, z-u>i) + u 2 £(t, z - u> 2 ). 

in the (u, 2;)-coordinates. Note that (27) is just gotten from the generalized theta 
function in 16 by letting t = t\ 2 tend to 0. 

In conclusion, one has h (X,M) = 1, hence there is a unique effective divisor 
5 = 5 which is the zero locus of a non-zero section of H°(X, M). 

As in the standard case, we parametrize an open subset of Pic°(S) with points 
in the union of P - U»=i,2 ; h=ioo U ^ with p t -(" u 7 Uf ") and p i ~ C U 0). We 
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can define the vertical singularities of the divisor S, whose equations, in the (u, z) 
coordinates, take the form 

£(t,z)=0, u-l£(t, z - uj-l) = 0, w 2 £(t, z - u> 2 ) =0, 

di£(T, z) + uidi£(T, z - wi) + wdiZir, z - w 2 ) =0, i = l,...,g-2. 

Again the vertical singular points of S are essentially of three types: 

(i) Consider a point x € D which is the image via ip : X — > X of a point in 
111,0 H U 20 — B, i.e. in the (u, z) coordinates one has U\ = u 2 = 0. Then this is a 
vertical singular point of S if and only if it corresponds to a singular point of S on 
IIi.o H II2.0 — B and to a singular point of 5b 1 +(, 2 on IIi j00 n II2.00 — B. 

(ii) Consider a point x € D which is the image via y of a point in II^o but not 
of a point in Ii3_j ; o, i.e. in the (u,z) coordinates one has u, = 0, 7^ 0, for an 
i = 1,2. If i = 1, this is a vertical singular point of S if and only if 

£(t,*) = 0, £(t,z-cc 2 ) =0, 
di£(r, z) + u 2 di£(T, z-w 2 )=0, i = l,...,g-2. 

i.e., if and only if z and z — b 2 arc in S and 7s(z) = 73(2 — 62)- Thus points of this 
type correspond to points in iVo(B, S). 

(hi) Consider a point x £ D, i.e., in the (u, z) coordinates one has u± 7^ 7^ 
u 2 . Then equations (28) mean that z corresponds to a point in S n n E(, 2 
where S, St 1 , St 2 are tangcntially degenerate. In other words points of this type 
correspond to points in N , 2 (B, S). 

Remark 17.2. A point x in Sing"(S) determines a quadric Q x in P s_1 . Remark 
17.1 is still valid here. 

A variant of this second type of rank-2 degeneration is obtained as follows. Given 
a G^-extension X = Xb of B determined by a point b — (61, 62) <= B 2 we consider 
two P 2 -bundles P and P' over B: 

¥ = ¥(O b ®L 1 ®L 2 ) and P' = P(L 2 © L x © {L x ® L 2 )), 

where we write as before Li for Lb t . We can glue these over the common P 1 - 
subbundle P(Li © L 2 ). Then we glue the P^subbundle ¥(O b © L{) of P with the 
P 1 -subbundle ¥(L 2 © (Li <g> L 2 )) of P' via a shift over 6 2 . Similarly, we glue the 
P^subbundle ¥{O b © £2) of P with the P^subbundlc P(Li © (Li <g> L 2 )) of P' via 
a shift over b\. In this way we obtain a non-normal variety over B. Both P and P' 
come with a relatively ample bundle Or (I) and CV(1). On P we have the linear 
equivalences 

III + 7T*(E 6l ) = II 2 + 7T*(S fc2 ) = IT3 + 7T*(S fcl+f)2 ). 

with IT, = ¥(O b © Li) for i = 1,2 and II 3 = P(ij © L 2 ). We let M be the line 
bundle C(n 3 +7r*(S fcl+f)2 )) on P and M' the line bundle 0{ll' 3 + 7r*(Sfc 1+i)2 ))on P', 
where IT 3 is the bundle P(Li © L 2 ). This descends to a line bundle M on X. This 
line bundle has a 1-dimensional space of sections generated by 

6(t, z) = £(r, z) + m £(t, z\ - u>i) + u 2 £(r, z - uj 2 ) 

in suitable affine coordinates (ui,u 2 ) on P 2 . Again the vertical singular points of 
S are essentially of three types: 
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(i) A point x e D which is the image via ip : X — ► X of a point in III n U 2 = 
P(Ob) — B is a vertical singularity if it corresponds to a singularity on S, to a 
singularity on on III n 1T3 and a singularity on E b2 on il 2 D II3; 

(ii) A point x e D which is the image via ^ of a point on one H3 but not of 
a point in LTi or LT2 is a vertical singular point of S if and only if x € l~l Sf, 2 
and 7h 6i (x) = "/E b2 {%)■ Thus points of this type correspond to points in Nq(B, S). 
Something similar happens for the points on exactly one of LTi or II2 . 

(hi) A point x ^ D is a vertical singularity if x e S n E^ n E b2 and S, , Sb 2 
are tangentially degenerate at x. In other words points of this type correspond to 
points in Nq^{B,E). 

We see that the compactification depends on a choice, but in both cases we can 
deal explicitly with the singularities of the theta divisors. 

Remark 17.3. The variant just given corresponds to a tesselation of M 2 given 
by the lines x = a, y = b and x + y = c with a, b, c G Z. To a triangle with 
integral vertices (n, m),(n + l,m) and (n, m + 1) (rcsp. (n — l,m), (n, m — 1) 
and (n,m)) we associate the P 2 -bundle ¥(L nbl+mb2 @ L( n+1)bl+mb2 ©£„&i+(m+i)& 2 ) 
(resp. P(i(„_i)6 1 +m6 2 ©^7161+^-1)62 ©£n6 1 +m& 2 ) and we glue the bundles belonging 
to adjacent triangles over the P 1 -bundle defined by the common edge. Then the 
generator t\ (resp. T2) of 1? acts by glueing the P 1 -bundle associated to an edge 
to the P 1 -bundle associated to the translate by x 1— > x + 1 (resp. y ^ y + 1) of this 
edge using a translation over & 2 (resp. &i). The quotient under 1? is the non- normal 
variety we just constructed. Also the earlier compctifications thus correspond to 
tesselations (see [26], §7). 

18. Compactification of A g 

In order to study the Andreotti-Mayer loci we need to compactify A g . The 
moduli space A g admits a 'minimal' compactification, the SataJce compactiGcation 
constructed by Satake and Baily-Borcl in characteristic and by Faltings-Chai over 
the integers (see [32], [13]). This compactification A* is an orbifold or stack which 
admits a stratification 

A* — I I 9 A- 

and the closure of A m in A* is A* n = \J^. Ai. This compactification is highly 
singular for g > 2. Smooth compactifications can be constructed by the theory 
developed by Mumford and his co-workers in characteristic and by Faltings-Chai 
in general. These compactifications depend on combinatorial data. We shall use 
the Voronoi compactification A g = A^° r as described by Namikawa, Nakamura and 
Alexeev (see [29, 28, 5]). This compactification is a smooth orbifold with a natural 
map q : A g — > A*. It has the stratification induced by that of ^4*: the stratum 

Ag ' = q {Ag — r) 

is called the stratum of quasi-abelian varieties of torus rank r. The general fibre of 

(r) 

q : A g — » Ag-r has a dense open subset consisting of standard compactifications as 
seenin§16. Hence the general fibre of q : A g r ^ — » A g - r has dimension gr—r(r+l) /2, 
thus dim(^ r)) ) = g(g + l)/2 - r. 

The closure of A g r) is A g - r) = q- 1 (A* g _ r ). We set dA g := A g - 1] , which is the 
boundary of A g . Moreover we set A g ^^ := A g — A g ^ r+1 \ 
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The fibres of the map q are well behaved if r < 4. Indeed, the 

points of A' g := A g - 4 ^ correspond to so-called stable quasi-abelian varieties which 
are compactifications of semi-abelian varieties, which can be explicitely described 
(see [29, 28, 5] and §§15, 17 for torus ranks 1 and 2). Thus one can define the vertical 
singular locus and the Andreotti-Mayer loci on the partial compactification A' g 
(seeRemark 18.1 below). For higher torus rank the situation is more complicated. 

For instance the fibres of q : A g ~ r ^ — > A*_ r might be non-reduced. However we 
will not need r > 4 here. 

An alternative approach might be to use the idea of Alexeev and Nakamura (cf. 
[1], [28], [5]) who have constructed canonical limits for one-dimensional families of 
abclian varieties equipped with principal theta divisors. 

The stable quasi-abelian varieties that occur in A' g for torus rank 1 and 2 are 
exactly those described in Section 15 and 17. For torus rank 3 these are described 
by the tesselations of R 3 occuring on p. 188 of [28], cf. also Remark 17.3. The open 
stratum of A g 3 ^ over A g -$ corresponds to the standard compactifications (see §16), 
obtained by glueing six P 3 -bundles over a {g — 3) -dimensional abelian variety B 
generalizing the construction for torus rank 2 where two P 2 -bundles were glued. 
These closed strata correspond to degenerations of the matrix T of the glueing 
data on which the standard compactifications depend (see §16). For instance the 
codimcnsion 3 stratum corresponds to the fact that in T two of the three elements 
above the main diagonal tend to zero (or to oo). 

Remark 18.1. As we remarked before, for stable quasi-abelian varieties corre- 
sponding to points {X, S) of A' g one can define the vertical singularities Sing vert (S) 
using ri 1 (log_D) as in the previous sections. One checks that for these compactifi- 
cations the analogue of Lemma 16.1 still holds. 

We will need the following result from [14]. 

Theorem 18.2. Let Z be an irreducible, closed subvariety of A g . Then Z D dA g 
is not empty as soon as codim^ (Z) < g. 

19. The Andreotti-Mayer loci and the boundary 

As we are working with a fixed compactification A g = A^° r and, as indicated in 
§18 above, we can define the Andreotti-Mayer loci over the part A' g = A g ~ 4 ^ of A g . 
We have N g ^ as a subscheme of A g and we define N g> k as the schematic closure. 
The support of N g ^ contains the set of points corresponding to pairs (X, S) such 
that Sing vort (AT) has a component of dimension at least k (sec [27]). 

It is interesting to look at the case k = 0, which has been worked out by Mumford 
[27] and Debarre [11]. In this case 7V 9j o is a divisor and by Theorem 18.2, every 
irreducible component N of this divisor intersects dA g . Let M be an irreducible 
component of N n dA g , which has dimension i^ 3 ^ 1 ) — 2. 

First of all, notice that M cannot be equal to Ag- 2 * 1 . This follows by the results 
in §17 and by Propositions 11.6 and 12.1. More generally, in the same way, one 

(V) 

proves that M cannot contain A g for any r = 2, 3 and 4. 

Hence M intersects A g in a non-empty open set of M, i.e., the intersection 
with the boundary has points corresponding semi-abelian varieties of torus rank 1. 
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If M does not dominate A g -i via the map q, then each fibre must have dimension 
g — 1. By Proposition 11.6 this implies that M dominates 7V~o, s -i. 

If M dominates A g -\ via q, the fibre of q\ M over a general point (B, S) G 
isJV (B,S). 

Recall now that Debarre proves in [11] that No, g has two irreducible components, 
one of which is the so-called theta-null component # ,g : the general abelian variety 
(X,Qx) m 9o, g , with Qx symmetric, is such that &x has a unique double point 
which is a 2-torsion point of X lying on © x ■ 

Let M 0j g be the other component. The general abelian variety (X, @x) in Mo, g , 
with Ox symmetric, is such that ©x has exactly two double points x and —x. It 
is useful to remind that, by Corollary 8.12, at a general point of either one of these 
component of N g ^ , the tangent cone to the theta divisor at the singular points is 
a smooth quadric. 

The component 9o, g intersects the boundary in two components, O'o !g ,@o !g , one 
dominating 9o, g -i, the other dominating A g -i with fibre over the general point 
(B,E) e A g -i given by the component 2S of N (B,E) (see §11). Also M 0j g inter- 
sects the boundary in two irreducible divisors M' , Mq g , the former is irreducible 
and dominates Mo lS _i, the latter dominates A g -i with fibre over the general point 
S) € A g -i given by the components of JVo(B, S) different from 2S. 

The main ingredient for Debarre's proof of the irreducibility of Mo i9 is a mon- 
odromy argument which implies that, if (B, 5) is a general principally polarized 
abelian variety of dimension g, then N (B, S) consists of only two irreducible com- 
ponents. 

Remark 19.1. Let (B, @b) be a general element in 6*o, 9 and let (X, S) be a semi- 
abelian variety of torus rank one with abelian part £?. Then there are no points in 
Sing vert (S) with multiplicity larger than 2. This follows from the fact that Ob has 
a unique singular point and by Remark 15.1. 

We finish this section with the following result which will be useful later on. It 
uses the notion of asymptotic cone given in §6. 

Proposition 19.2. One has: 

(i) let g > 3, let (B, 5) be a general point o/#o,g o,nd let x be the singular point 
of 5. Then the asymptotic cone is strictly contained in the quadric 
tangent cone Q x ; 

(ii) let g > A, let (B,E) be a general point of M , g and let x, —x be the singular 
points of 5. Then the asymptotic cone 

TC (3) 

is strictly contained in the 

quadric tangent cone Q x = Q- x - 
Proof. Degenerate to the jacobian locus and apply the results from [18, 19]. □ 

20. The Conjecture for N\ t9 

In this section we prove Conjecture 9.1 for k = 1. We consider an irreducible 
component N of 7V 9: i which is of codimension 3. The first observation is that 
the assumption about the codimension of TV implies that the generic principally 
polarized abelian variety is simple since by Remark 7.4 every irreducible component 
of A™ s has codimension > g — 1 > 3 if we assume g > 5. 

Proposition 20.1. Let g > 6 and let N be an irreducible component of N gt \ which 
is of codimension 3 in A g . Then N intersects the stratum Ag 1 ^. 
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Proof. We begin by remarking that N cannot be complete in A g in view of Theorem 
18.2. Therefore N intersects dA g . Since dA g is a divisor in A g it intersects N in 
codimcnsion one. Let M be an irreducible component of Nf]dA g . It is our intention 
to prove that M has a non-empty intersection with Ag 1 ^ . 

Suppose that M C A g ^ . For dimension reasons we have M = A g ^ . Since we 
are using a compactification A g such that the general point of A g ^ corresponds to 
a standard compactification {X, S) of torus rank 4 with abelian part (B, S) € A g -4 
we deduce from Lemma 16.1 and Remark 18.1 that if dim(Sing vcrt (S)) > 1 then 
(B, S) G N g -4.o- But for g > 5 the locus -/V 9 -4,o is a divisor in A 9 -a and we obtain 
the inequality codim^ (M) > codim^ (*4 S 4 ^) + 1 > 5, a contradiction. Therefore 

we can assume that M n Ag ^ 0. 

Suppose that M n has codimcnsion 1 in _4 S . Then either M maps domi- 

~ (3} ~ 

nantly to A g -z via the map q : A g ---» -4 s -3 and M intersects the general fibre 
F of q in a divisor, or M maps to a divisor in -4 s -3 under q with full fibres F 
contained in M. 

The former case is impossible by Proposition 12.1. In the latter case for a general 
(B, S) in g'(M) all the quasi-abelian varieties (X, S) in the fibre F over (£?, S) must 
have a 1-dimensional vertical singular locus of S. Note that (X, H) corresponds to 
a standard compactification as considered in §16. By the discussion given in §16 
and by Proposition 11.6, we see that q(M) has to be contained in N g s t i, against 
Theorem 8.6. We thus conclude that M n A ( g - 2) ^ 0. 

Suppose that M n A g ~ 7^ which has codimension 2 in *4g . As above we have 
that q{M) is contained in N g _2,o- 

Suppose first g(M) is dense in a component of N g _2,o- If is a general 

( 2) ~ 

element of q(M), then M intersects the fibre of g : .4 g ; --■» -4 9 -2 over in 
codimcnsion one. This gives a contradiction by the analysis §17 and Proposition 
11.6. 

Suppose that q(M) is not dense in a component of N g _2,o- If (B, S) is a general 
element of q(M), then M contains the full fibre of g : ,4 S — > -^9-2 ovcr (B,^)- 
By taking into account the analysis §17 and Proposition 11.6, this implies q(M) 
contained in Ng-2,1, giving again a contradiction. 

This proves that M n A g - 1] ^ 0. □ 



Let g > 4 and let AT be an irreducible component of N gt \ of codimension 3 in A g . 
As in the proof above we denote by M an irreducible component of the intersection 
of the closure of N in A g with the boundary dA g . According to Lemma 20.1 the 
morphism q : A g — > A* induces a rational map a : M — > -4 s -i, whose image is 
not contained in dA g -i- 

Lemma 20.2. In the above setting, the Zariski closure of q(M) in A g -i is: 

(i) either an irreducible component Ni of N g -i,i of codimension 3 in A g -i; 

(ii) or an irreducible component Nq of N g -i t o and in this case: 

(a) iff] = (B,E) ^ Nq is a general point, then the closure of <z _1 (??) in B 
is an irreducible component of Ni(B,E) of codimension 2 in B; 
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(b) if £ = (X,E) e M is a general point, then Sing vort (S) meets the 
singular locus D of X in one or two points, whose associated quadric 
has corank 1. 

Proof. If q(M) C A^_i,i, then Theorem 8.6 implies 3 < codim Ag _ 1 (q(M)) < 
codhn^ (M) = 3 and the closure of q(M) must an irreducible component of 

Ng-i,i. If q{M) % N g _ 1A then by Proposition 12.1 we have that q{M) C N g _ lfi 
and the fibre q^ 1 (B, S) C Ni(B,S). By Proposition 11.6 we have codiniB(Ai(i?, S)) > 
2 and since -/V 9 _i, is a divisor in *4 9 -i we see that (ha) follows. The last statement 
(iib) follows from Remark 12.3, the analysis in Section 17, the description of A gj0 
by Mumford and Debarre (see [27], [11], and §19) and Corollary 8.12. □ 

We are now ready for the proof of the conjecture for N g ^. 

Theorem 20.3. Let g > 4. Then the codimension of an irreducible component N 
of N g _i in A g is at least 3 with equality if and only if: 

(i) g — 4 and either N = H4 is the hyperelliptic locus or N = .44,(1.3); 

(ii) g = 5 and N = J 5 is the jacobian locus. 

Proof. By Theorem 8.6, the codimension of N is at least 3. Suppose that TV has 
codimension 3. It is well known that the assertion holds true for g = 4 and 5 (see 
[4], [10], [8]). We may thus assume g > 6 and proceed by induction. 

Let £ = (X, Qx) be a general point of N, so that X is simple (see Remark 7.4). 
Let S be a 1-dimensional component of Sing(Ox). We can assume that the class 
of S in X is a multiple iwyx of the minimal class -fx = @jT /(# — 1)' *= H 2 (X, Z). 
If not so, then End(X) ^ Z and this implies that codim^ (N) > g — 1 (see Remark 
7.4). 

By Theorem 8.6, the general point in S is a double point for Qx- We let R 
be the curve in S g whose general point is £ = (X, Qx,x), with x € S a general 
point. Note that R is birationally equivalent to S. Let Q be the linear system of 
all quadrics in P(Tx,o)- One has the map 

(f>:£e R--+Q,; e Q. 

As in the proof of Theorem 8.6, the map 4> is not constant. Let Qr be the span of 
the image of <f>. As in the proof of Theorem 8.6, one has dim(Q^) > 2. By Propo- 
sition 8.3, Qr is contained in the linear system 7V 9i i(C) (see §7 for the definition), 
which has dimension at most 2 since codim_4 g (A r ) = 3. Thus Qr = J\f gi i(() has 
dimension 2. 

By Lemma 20.1, the closure of N in Ag~^ has non empty intersection with the 
boundary. As in the proof of Lemma 20.1, we let M be an irreducible component 
of the intersection of the closure of N in Ag" 1 "* with the boundary. Consider the 
rational map a : M --■» A g -\ and the closure of the image a(M), for which we 
have the possibilities described in Lemma 20.2. 

Claim 20.4. Possibility (i) in Lemma 20.2 does not occur. 

Proof of the claim. By induction, one reduces to the case g = 6 and a(M) — J§. 
Let (A, S) G M be a general point. Then (A", H) is a general rank one extension 
of the jacobian (J(C), 0c) of a general curve C of genus 5. Note that if a; € J(C) 
corresponds to the extension, then Qc and x + @c are not tangentially degenerate 
(see [17], Thm. 10.8, p. 273). Then the analysis of §15 implies that the vertical 
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singular locus So of S sits on the singular locus D = J{C) of X and it is isomorphic 
to Sc = Sing(9c) with cohomology class 6 4 /12 (see [3]). Thus S • S a = Oc • Sc — 
10. Hence, if ( = (X,Ox) is a general point of N, then Sing(6x) is a curve S 
such that Ox ■ S = 10. On the other hand S is homologous to mj x and one has 
10 = mOx ■ lx — 6m, a contradiction. □ 

Claim 20.4 shows that only possibility (ii) in Lemma 20.2 can occur. In partic- 
ular, by (iib) of Lemma 20.2, for £ = (X, 9x,i) general in R, the quadric has 
corank 1. Let £ P 9_1 be the vertex of Q^. Remember that R is birational to S. 
Hence, by Proposition 4.4, the map 

7 : £_ £ R — > v e £ P 9 - 1 

can be regarded as the Gauss map 75 of S. 

Claim 20.5. If the general quadric in the linear system Qr is singular, then for 
£ = (X, @x , x) general in R, the vertex of is contained in the asymptotic 
cone TCj^ . 

Proof of the claim. Suppose the general quadric in Qr is singular. Then the general 
quadric in Qr has corank 1 (see Lemma 20.2, (iib)) and, by Bertini's theorem, its 
vertex lies in the base locus of Qr. In particular, for £ = (X, Ox, x) general in R, 
the vertex of the quadric lies in all the quadrics of Qr. 

Choose a local parametrization x = x{t) of S around a general point of it, with 
t varying in a disc A. Then £(t) = (X,Qx,x(t)) £ R and we set Q$(t) '■— Qt, its 
equation being 

y^ j d i dj6{x{t))z i Zj = 0, 

ij 

where we set 9(z) := 0{t, z) for the theta function of X . The main remark is that 
all the subsequent derivatives of Qt with respect to t lie in Qr and actually Q t and 
its first two derivatives Q' t and Q" span Qr, because dim(Q^) = 2. Hence Bertini's 
theorem implies that x' := x'(s) sits on all these quadrics for t and s general in A. 
The equations of Q' t and Q" are respectively 

Y J d t d J d h 6(x{t))x' h (t)^j 

ijh 

^didjdhdkOixityx'^ty^ZiZj +^9id j d h 6{x{t))xl{t)z i z j = 

ijhk ijh 

Thus we have the relations 
(29) 

Y,d l d 3 9{x(t))x' l {s)x' 1 {t) = 0, Y, d ^ dhd ^ x 'h(tWMx' J (s) = 

ij ijh 

^didjdhdkeix^x'^x'^x'Mxjis) +Y,didjd h e{x{t))x'l l {t)x' i {s)x' j {s) = 

ijhk ijh 

identically in s,t £ A. The first of these relations says that the tangent hyperplane 
to Qt at x'(t) contains the vertex x'(s). From the second relation we have 

(so) Y, d ^ d ^ x ^(tH(ty,(t) = 

ijh 
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(3) 

which shows that is contained in the asymptotic cone TCf J . By differentiating 
(30), one finds 

]T Sia^flifefl^wj^w^w^w^w + 3j2d i d j d h e(x(t)K(t)x> i (t)x' j (t) = o. 

ijhk ijh 

By comparing with (29) for s — t, we deduce that 

ijhk 

which proves that v^eTC^K □ 

The crucial step in our proof is the following claim. 

Claim 20.6. The general quadric in the linear system Qr is non singular. 

Proof of the claim. Suppose this is not the case. Again we consider an irreducible 
component M of N n dA' g and let (X,S) be a general point of M. By Claim 20.4 
and Lemma 20.2, {X, E) is a rank 1 extension of (B, 5) corresponding to a general 
point in a component of N g -i t o, with extension datum b varying in a codimension 
2 component of Ni(B,E). We let So be the vertical singular locus of 5. By the 
analysis of §15, this corresponds to a contact curve C := C% of 5 with S&, which 
contains the singular points of both S and (see Remark 12.3). This means that 
we have a point on So, corresponding to a singular point x of S, where the tangent 
cone is the cone over Q x with vertex the point of P ff_1 corresponding to b (see S 
15). 

Now we note that C is smooth at x. Indeed locally around x, the divisor S 
looks like a quadric cone of corank 1 in P 9 ~ 2 and looks like a hyperplane, which 
touches it along a curve. This implies that C locally at x looks like a line along 
which a hyperplane touches a quadric cone of corank 1 (see Remark 12.4). 

Hence, the Gauss image Xb := "fc b {x) lies in Q x and actually, by Claim 20.5, the 
point Xb lies in the asymptotic cone TC X ^ . 

The Gauss map 7s : S — ■ > P ff_2 of 5 has an indeterminacy point at ir and 
— x, which can be resolved by blowing up x and —x, since we may assume S to 
be symmetric. Let p : E — > 5 be the blow-up and let 7h be the morphism which 
coincides with 73 o p : 5 — - > P 9 ~ 2 on an open subset. The exceptional divisor at 
x and —x is isomorphic to Q x and 7h(^&) is the tangent hyperplane to Q x at Xb- 
The tangency property of S and E b along C implies that the tangent hyperplane 
to Q x at Xb coincides with -f~ b (x) . 

Now we let b vary in a component Z of N\(B, S) of dimension 5 — 3, so that we 
have a rational map 

/ : Z — > Q x , 6 1 ^ x fc . 

Note that Qa: also has dimension 3 — 3 and we claim that / has finite fibres, hence 
it is dominant. If not, we would have an irreducible curve r in Z such that for all 
b e L, Xb stays fixed. But then for the general b e T, the divisor Sf, has a fixed 
tangent hyperplane at x, a contradiction by Lemma 11.1. 

On the other hand, by Claim 20.5 and by part (i) of Proposition 19.2, one has 
that / cannot be dominant, a contradiction. □ 

By Lemma 20.6, the curve 4>(R) := £ is an irreducible component of the dis- 
criminant A C Qr of singular quadrics in Q R = F 2 . Note that A has degree g in 
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P 2 . The map <f> has degree at least 2 since we may assume ©x to be symmetric, 
hence it factors through the multiplication by -lj on X. 

Claim 20.7. The map <p : R^Y* has degree 2. 

Proof. Let d > 2 be the degree of the map. Then for £ = (X, Ox, a;) general in 
R, we have distinct points & = (X,Qx,%i) € R, with £ = £i, such that all the 
quadrics Qg 4 , i = 1, . . . , d, coincide with the quadric Q = Q^. If for i = 1, . . . , d we 
let ?7j be the tangent vector to S at Xi, we have that ??i = • • • = % and d Vi Q^, for 
i = 1, . . . , d, coincide with the quadric Qi = c^Q, which is linearly independent 
from Q. The analysis we made in §3 shows that S g is smooth at each of the points 
£i, « = 1, . . . ,d, and the tangent space there is determined by Q and Q m - This 
implies that a general deformation of £ = (X, ©x , x) inside S g carries with it a 
deformation of each of the points & (i — l,...,d) in S g , because the involved 
quadrics are the same at these points. This yields that a general element of an 
irreducible component of iV Sj0 containing N has at least d singular points. By 
Debarre's result in [11] (sec §19), one has d = 2, proving our claim. □ 

Claim 20.8. The map <p is a morphism. 

Proof. To prove the claim, it suffices to show that N is not contained in N g ^, r , 
with r > 3. In order to prove this, one verifies that, for a general point {X, S) in 
a component M of the intersection of TV with the boundary, there are no points of 
multiplicity r > 3 in Sing vert (S). Recall that, by Proposition 20.1, we may assume 
that (X, S) is a semi-abelian variety of torus rank 1, with abclian part r\ = (B, S). 
Moreover, by Claim 20.4, only case (ii) of Lemma 20.2 can occur. Therefore we 
may assume that r\ is either a general point of 0o,g-i or a general point of Mo lS _i 
and the extension corresponds to a general point in an irreducible component of 
Ni(B,E) which has dimension g — 3 > 0. By Remark 15.1 we see that no triple 
points can occur on Sing vcrt (S). □ 

Note now that the morphism <f> is defined on R = S by sections of 0s (Ox), 
since the points of S verify the equations (2) and, if C = (X, ®x,x) £ R, the entries 
of the matrix of are didj6(r,z), where z corresponds to x. We deduce from 
deg(A) = g and from Claim 20.7, that 



As we assumed at the beginning of the proof, the class of S in X is a multiple m-fx 
of the minimal class. In view of (31), we find m < 2. The Matsusaka-Ran criterion 
[31] and a result of Welters [39] imply that (X, Ox) is either a Jacobian or a Prym 
variety. Since g > 6 this is not possible in view of the dimensions. This ends the 



Remark 20.9. A. Verra communicated to us an interesting example of an irre- 
ducible component M of codimension 6 of A^i contained in the Prym locus. We 
briefly sketch, without entering in any detail, its construction and properties. Let C 
be the normalization of a general curve of type (4, 4) on P 1 x P 1 with two nodes on 
a line of type (0, 1), so that C has genus g = 7. Let d, t be the linear series formed 
by pull-back divisors on C of the rulings of type (1, 0), (0, 1) respectively. Consider 
a non trivial, unramified double cover / : C — > C of C and let (P, S) be the corre- 
sponding Prym variety. Then 5 has a 1-dimensional unstable singular locus R (see 
[23]), homologically equivalent to twice the minimal class, described by all classes 



(31) 



S ■ Ox < 2g. 



proof. 



□ 
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in Pic 12 (C) of divisors of the form f*(d) + M, with f*(M) e t. One proves that the 
map 4> described in the proof of Theorem 20.3 sends R to a plane sextic of genus 
7 which is tetragonally associated to C (see [23] for the tetragonal construction). 
The divisor S has 24 further isolated singular points, which are pairwise exchanged 
by the multiplication by —1. One shows that the corresponding 12 tangent cones 
span the same linear system Q of dimension 2 spanned by (f>(R) ■ The linear system 
Q is the tangent space to the Prym locus Ve at (P, S), which is therefore a smooth 
point for V§. By contrast, M is a non-reduced component of iVg^ of codimension 
6 such that the projectivized normal space 2 at a general point has dimension 2 
rather than 5. This shows that the hypotheses of Theorem 20.3 cannot be relaxed 
by assuming only that the projectivized normal space to M at a general point has 
dimension 2. 

21. Appendix: A Result on Pencils of Quadrics 

One of the ingredients of the proof Theorem 8.6 is a classical result of Corrado 
Segre from [33] on pencils of quadrics. 
First we recall the following: 

Proposition 21.1. Let C be a pencil of quadrics in P" with n > 1 whose general 
member Q is smooth. Then: 

(i) the number of singular quadrics Q € C is n + 1, where each such quadric 
Q has to be counted with a suitable multiplicity n(Q) > n + 1 — rk(Q); 

(ii) for a singular quadric Q € C one has fi(Q) > 2 if and only if either rk(Q) < 
n or the singular point of Q is also a base point of C; 

(hi) for a singular quadric Q € C with rank n one has n(Q) — 2 if and only if 
any other quadric Q' G C is smooth at p and the tangent hyperplane to Q' 
at p is not tangent to Q along a line. 

Proof. Consider the linear system Q n of dimension n(n + 3)/2 all quadrics in P n . 
Inside Q n we have the discriminant locus A„ of singular quadrics, which is a hy- 
persurface of degree n + 1, defined by setting the determinant of a general quadric 
equal to zero. The differentiation rule for determinants implies that the locus A„, r 
of quadrics of rank r < n + 1 has multiplicity n + 1 — r for A„. By intersecting A„ 
with the line corresponding to C we have (i) . 

As for assertion (ii), we may assume rk(Q) = n, so that Q has a unique double 
point p, which we may suppose to be the point (1, 0, . . . , 0). Thus the matrix of Q 
is of the form 

'0 0„ 

where 0„ € C" is the zero vector and A is a symmetric matrix of order n and 
maximal rank. Let Q' be another quadric in £, with matrix 

[3 b 
b l B 

with f3 e C, b = (bi, . . . , b n ) e C" and B is a symmetrix matrix of order n. By 
intersecting C with A„, we find the equation 

^ det (^ A + tB )=°- 
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The constant term in the left hand side is 0. The coefficient of the linear term is 

det^ \ )=/?det(A) 

which proves (ii). 

Let us prove (iii). Suppose rk(Q) = n, so that Q has a unique double point p, 
which is a base point of C. Again we may suppose p is the point (1 : : . . . : 0) 
and we can keep the above notation and continue the above analysis. The left hand 
side in (32) is 

hence the coefficient of the third order term is 
(33) det(° * ). 

One has/z(Q) = 2 if and only if this determinant is not zero, hence 6^0, which is 
equivalent to saying that all quadrics in the pencil different from Q are smooth at 
p. Note that there is a vector c = (ci, . . . , c n ) <E C™ such that b = c ■ A. Now the 
determinant in (33) vanishes if and only if c • 6* = 0, i.e. c • A ■ c* = 0. This means 
that the line L joining p with (0 : ci : . . . : c„) sits on Q and that the tangent 
hyperplane to Q' at p, which has equation b\Xi + ■ + b n x n = 0, is tangent to Q 
along r. □ 

Next we prove Segre's theorem. 

Theorem 21.2. Let £ be a linear pencil of singular quadrics in P" with n > 2 
whose general member Q has rank n + l — r, i.e. Vert(Q) = P r_1 . We assume that 
Vert(Q) is not constant when Q varies in C with rank n + l — r. Then: 

(i) the Zariski closure 

Vc = ( |J Vert(Q)) 

Q£C,rk{Q)=n+l-r 

is a variety of dimension r spanning a linear subspace IT of dimension m 
in P™ with r<m<(n + r-l)/2; 

(ii) Vc is a variety of minimal degree m — r + 1 in IT; 

(iii) if 

dim( p| Vert(Q)) = s 

QeC,rk(Q)=n+l-r 

then r <(n + 2s + 3)/3; 

(iv) the number of quadrics Q G C of rank rk(Q) < n+l — r is n + r — 2m — 1 < 
n — r — 1, where each such quadric Q has to be counted with a suitable 
multiplicity v{Q) > n + 1 — r — rk(Q). 

Proof. We start with the proof of part (i). Notice that, by iteratedly restricting to 
a general hyperplane, we can reduce to the case r = 1. 

In this case Vc is a rational curve which, by Bertini's theorem is contained in the 
base locus B of C. Let p, q be general points on it and let L be the line joining them. 
There is a quadric Q p € C with vertex at p. Hence Q p contains L. Similarly there 
is a different quadric Q q with vertex at q, and it also contains L. Since Q p and Q q 
span £, we see that L is contained in B, i.e., the secant variety to Vc is contained 
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in B. Take now three general points p,q,r on Vc- Since the lines pq, pr, qr are 
contained in B also the plane spanned by p, q, r is contained in B. Continuing this 
way, we see that IT = (Vc) is contained in B. Since the general quadric in C has 
rank n (recall we are assuming r = 1 now), the maximal dimension of subspaces on 
it is n/2. Thus dim(il) < n/2 which proves part (i). 

Also for (ii) we can reduce ourselves to the case r = 1, in which we have to prove 
that Vc is a rational normal curve in II = (Vc)- Set dim(IT) = m. 

Let p e Vc be a general point. The polar hyperplanc ir p of p with respect to 
Q e C does not depend on Q, since there is a quadric in C which is singular at 
p (see the proof of Proposition 21.1). Note that ir p has to contain all vertices of 
the quadrics in C, hence it contains II. By the linearity of polarity, we have that 
polarity with respect to all quadrics in C is constant along II and for a general 
point x € IT, the polar hyperplane n x with respect to all quadrics in C contains IT. 
Furthermore the linear system of hyperplanes V = {tt x } x ^u has dimension m — 1. 

Now, let p e Vc be a general point and let Q p be the unique quadric in C with a 
double point at p. We denote by Star(p) the P m_1 of all lines in IT containing p. Let 
7r e V be a general hyperplanc, which is tangent to Q p along a line L containing p. 
Moreover L sits in IT, because this is the case if n — ir q with q another general point 
on Vc, in which case L is the line (p, q). Thus we have a linear map (j> p : V — > Star(p), 
which is clearly injective and therefore an isomorphism. 

Fix now another general point q on Vc- The two maps <j) p and (j) q determine a 
linear isomorphism <f) ■ Star(p) — > Star(g). Note that L meets <f>(L) if and only if 
L n 4>(L) is a point of Vc- This implies that Vc is a rational normal curve in IT, 
proving part (ii). 

Let us prove part (iii). It suffices to prove the assertion if s = — 1. The variety 
Vc is swept out by a 1-dimensional family of projective spaces of dimension r — 1, 
i.e., the vertices of the quadrics in C. Under the assumption s = — 1 no two of 
these vertices can intersect. Thus we must have 2(r — 1) < m. Using part (i), the 
assertion follows. 

Finally we come to part (iv). Let us restrict £ to a general subspace A of 
dimension n — r. We get a pencil C of quadrics in A whose general member is 
smooth. 

We get a singular quadric in C when we intersect A with a quadric in C whose 
vertex intersects A. We claim that this is the only possibility for getting a singular 
quadric in C. Indeed, let Q e C and suppose that its intersection Q £ C with A 
is singular at p e A, but Q is not singular at p. Then A is tangent to Q at p and 
therefore also intersects the vertex of Q. 

In conclusion we have only two possibilities for getting singular quadrics in C: 

(a) there is quadric of rank n + 1 — r in C whose vertex intersects A; 

(b) there is quadric of rank — h <n+l — r in C giving rise to a quadric 
of rank n — h in L. 

Case (a) occurs as many times as the degree of Vc, that is, m — r + 1 times. 
According to part (ii) of Proposition 21.1, each quadric Q in case (a) contributes 
with multiplicity at least 2 in the counting of singular quadrics in L. We claim 
that, because of the generality of A, this multiplicity is exactly 2. To prove this, by 
part (iii) of Proposition 21.1, we will prove that for each quadric Q in case (a), with 
vertex p £ A and for any other quadric Q' e C, the tangent hyperplane ir to Q' at p 
is not tangent to Q along a line contained in A. To see this we can, by first cutting 
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with a general subspace of dimension n — r + 1 through A, reduce ourselves to the 
case r = 1, in which Vc is a rational normal curve. Choose then a general point 
q € Vc and let Q' = Q q be the unique quadric in C with a double point at q. The 
hyperplane 7r is tangent to Q q along the line (p, q) . This implies that 7r is tangent 
to Q only along the tangent line L p to Vc at p (see the proof of part ii)). By the 
generality assumption, A is not tangent to Vc at p. Thus the assertion follows. 

As for quadrics in case (b), again by part (i) of Proposition 21.1, each such 
quadric contributes to the same count with multiplicity h — r. Since, by part (i) of 
Proposition 21.1, the number of singular quadrics in £, counted with appropriate 
multiplicity, is n — r + 1, the assertion follows. □ 

One has the following consequence: 

Corollary 21.3. Let £ be a linear pencil of quadrics in P™ with n > 2. Then the 
general member Q e £ has rank n + 1 — r (i.e., Vert(Q) = P r ~ 1 ) if and only if 
the base locus of £ contains a linear subspace II of dimension m with r < m < 
(n + r — l)/2, along which all the quadrics in £ have a common tangent subspace 
of dimension n + r — m — 1. In this case IT is the span of the variety Vc- 

Proof. As usual it suffices to prove the assertion for r = 1. If the general quadric 
Q E £ has rank rk(Q) = n, the assertion follws from the proof of Theorem 21.2. 
The converse is trivial, since a smooth quadric in P n has a tangent subspace of 
dimension n — m — 1, and not larger, along a subspace of dimension m. □ 

These results imply the existence of canonical forms for pencils of singular 
quadrics, originally due to Weierstrass [37] and Kronecker [21]. This is explained 
in some detail in [33], §§20-25, and we will not dwell on this here. 

It would be desirable to have an extension of the results in this Appendix to 
higher dimensional linear families of quadrics. 
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